Max level shown:
12025-06-04T01:30:01.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a189bfbc-8e93-48aa-bc9a-b12eb7558a03 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22025-06-04T01:30:01.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a189bfbc-8e93-48aa-bc9a-b12eb7558a03 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32025-06-04T01:30:01.926ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a189bfbc-8e93-48aa-bc9a-b12eb7558a03 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
42025-06-04T01:30:02.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 539833cb-1e91-4ec0-99c5-3e77b0a176c8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
52025-06-04T01:30:02.946ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 539833cb-1e91-4ec0-99c5-3e77b0a176c8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
62025-06-04T01:30:02.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 539833cb-1e91-4ec0-99c5-3e77b0a176c8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
72025-06-04T01:30:03.670ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0bd83325-7fa3-44fa-9be6-03af29fad3dd uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-06-04T01:30:03.669844159Z }
82025-06-04T01:30:03.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0bd83325-7fa3-44fa-9be6-03af29fad3dd roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
92025-06-04T01:30:03.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0bd83325-7fa3-44fa-9be6-03af29fad3dd resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
102025-06-04T01:30:03.678ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 8833 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0bd83325-7fa3-44fa-9be6-03af29fad3dd response_code = 204 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
112025-06-04T01:30:04.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2e786134-f504-40a4-9383-3d42ec505ca8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
122025-06-04T01:30:04.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2e786134-f504-40a4-9383-3d42ec505ca8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
132025-06-04T01:30:04.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2e786134-f504-40a4-9383-3d42ec505ca8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
142025-06-04T01:30:05.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1654a663-387d-4cd2-9129-453f25602966 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
152025-06-04T01:30:05.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1654a663-387d-4cd2-9129-453f25602966 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
162025-06-04T01:30:05.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1835 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1654a663-387d-4cd2-9129-453f25602966 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
172025-06-04T01:30:06.136ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2140 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-06-04T01:30:03.679269423Z }
182025-06-04T01:30:06.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
192025-06-04T01:30:06.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
202025-06-04T01:30:06.145ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e vmm_needs_update = true
212025-06-04T01:30:06.145ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
222025-06-04T01:30:06.145ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
232025-06-04T01:30:06.147ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
242025-06-04T01:30:06.147ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
252025-06-04T01:30:06.149ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
262025-06-04T01:30:06.149ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
272025-06-04T01:30:06.152ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
282025-06-04T01:30:06.152ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
292025-06-04T01:30:06.154ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
302025-06-04T01:30:06.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
312025-06-04T01:30:06.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
322025-06-04T01:30:06.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Database result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
332025-06-04T01:30:06.157ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
342025-06-04T01:30:06.160ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
352025-06-04T01:30:06.167ZINFOnexus (dropshot_internal): starting update saga for 5abad387-28b5-4eac-817c-81deb58a742d actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1603 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-06-04T01:30:03.679269423Z }
362025-06-04T01:30:06.167ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 31003 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 response_code = 204 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
372025-06-04T01:30:06.169ZDEBGnexus (dropshot_internal): preparing instance-update saga for 5abad387-28b5-4eac-817c-81deb58a742d... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 5abad387-28b5-4eac-817c-81deb58a742d local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 49d957ca-ced2-4343-8f87-faabeec35a39 uri = /vmms/a8435d87-6398-4e8f-9fda-269bcd96ac6e
382025-06-04T01:30:06.169ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update
392025-06-04T01:30:06.169ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"5abad387-28b5-4eac-817c-81deb58a742d","lookup_type":{"ById":"5abad387-28b5-4eac-817c-81deb58a742d"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ById":"6a575fb0-c195-4d2a-ac26-16b83f49e832"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
402025-06-04T01:30:06.169ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update
412025-06-04T01:30:06.214ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update
422025-06-04T01:30:06.214ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
432025-06-04T01:30:06.216ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
442025-06-04T01:30:06.216ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
452025-06-04T01:30:06.224ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
462025-06-04T01:30:06.224ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
472025-06-04T01:30:06.227ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
482025-06-04T01:30:06.227ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
492025-06-04T01:30:06.235ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
502025-06-04T01:30:06.235ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("e755593d-8ac7-4b40-9f18-b1abd5702045")) node_id = 0 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
512025-06-04T01:30:06.238ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
522025-06-04T01:30:06.238ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
532025-06-04T01:30:06.250ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update
542025-06-04T01:30:06.255ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
552025-06-04T01:30:06.255ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
562025-06-04T01:30:06.258ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
572025-06-04T01:30:06.258ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
582025-06-04T01:30:06.260ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
592025-06-04T01:30:06.260ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
602025-06-04T01:30:06.264ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
612025-06-04T01:30:06.265ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
622025-06-04T01:30:06.266ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
632025-06-04T01:30:06.266ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
642025-06-04T01:30:06.267ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
652025-06-04T01:30:06.267ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
662025-06-04T01:30:06.269ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
672025-06-04T01:30:06.272ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
682025-06-04T01:30:06.272ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance updater_id = e755593d-8ac7-4b40-9f18-b1abd5702045
692025-06-04T01:30:06.272ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
702025-06-04T01:30:06.272ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
712025-06-04T01:30:06.285ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance updater_id = e755593d-8ac7-4b40-9f18-b1abd5702045
722025-06-04T01:30:06.285ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
732025-06-04T01:30:06.285ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance
742025-06-04T01:30:06.292ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 5abad387-28b5-4eac-817c-81deb58a742d locked_gen = Generation(Generation(2)) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = LockInstance updater_id = e755593d-8ac7-4b40-9f18-b1abd5702045
752025-06-04T01:30:06.297ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
762025-06-04T01:30:06.297ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("e755593d-8ac7-4b40-9f18-b1abd5702045")}) node_id = 1 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
772025-06-04T01:30:06.301ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
782025-06-04T01:30:06.301ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
792025-06-04T01:30:06.309ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
802025-06-04T01:30:06.309ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
812025-06-04T01:30:06.311ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
822025-06-04T01:30:06.311ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
832025-06-04T01:30:06.312ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
842025-06-04T01:30:06.312ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
852025-06-04T01:30:06.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e784dea-476c-4da0-b514-d15b902d43f8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
862025-06-04T01:30:06.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e784dea-476c-4da0-b514-d15b902d43f8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
872025-06-04T01:30:06.314ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
882025-06-04T01:30:06.315ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1690 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e784dea-476c-4da0-b514-d15b902d43f8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
892025-06-04T01:30:06.317ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
902025-06-04T01:30:06.317ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
912025-06-04T01:30:06.317ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
922025-06-04T01:30:06.323ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: a8435d87-6398-4e8f-9fda-269bcd96ac6e, time_created: 2025-06-04T01:29:45.199557Z, time_deleted: None, instance_id: 5abad387-28b5-4eac-817c-81deb58a742d, sled_id: 4adf0198-0a8a-47c8-a668-dc22afd01e24, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-06-04T01:30:03.679269Z, gen: Generation(Generation(7)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-06-04T01:29:43.103614Z, gen: Generation(Generation(3)), propolis_id: Some(a8435d87-6398-4e8f-9fda-269bcd96ac6e), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(a8435d87-6398-4e8f-9fda-269bcd96ac6e (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-06-04T01:30:06.323474108Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
932025-06-04T01:30:06.328ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
942025-06-04T01:30:06.328ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"5abad387-28b5-4eac-817c-81deb58a742d","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"a8435d87-6398-4e8f-9fda-269bcd96ac6e"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"5abad387-28b5-4eac-817c-81deb58a742d","lookup_type":{"ById":"5abad387-28b5-4eac-817c-81deb58a742d"},"parent":{"key":"6a575fb0-c195-4d2a-ac26-16b83f49e832","lookup_type":{"ById":"6a575fb0-c195-4d2a-ac26-16b83f49e832"},"parent":{"key":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","lookup_type":{"ById":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"e755593d-8ac7-4b40-9f18-b1abd5702045"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"6a575fb0-c195-4d2a-ac26-16b83f49e832","ram_diff":1073741824},"destroy_active_vmm":"a8435d87-6398-4e8f-9fda-269bcd96ac6e","destroy_target_vmm":null,"network_config":"Delete","new_intent":null,"new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-06-04T01:30:06.323474108Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
952025-06-04T01:30:06.328ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
962025-06-04T01:30:06.362ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
972025-06-04T01:30:06.362ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
982025-06-04T01:30:06.362ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
992025-06-04T01:30:06.362ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1002025-06-04T01:30:06.365ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1012025-06-04T01:30:06.365ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1022025-06-04T01:30:06.368ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1032025-06-04T01:30:06.368ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1042025-06-04T01:30:06.371ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1052025-06-04T01:30:06.371ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("a7bdc6e6-6858-4b38-be73-f70160601be1")) node_id = 0 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1062025-06-04T01:30:06.375ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1072025-06-04T01:30:06.375ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1082025-06-04T01:30:06.378ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 5abad387-28b5-4eac-817c-81deb58a742d parent_lock = UpdaterLock { updater_id: e755593d-8ac7-4b40-9f18-b1abd5702045, locked_gen: Generation(Generation(2)) } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1092025-06-04T01:30:06.378ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = BecomeUpdater
1102025-06-04T01:30:06.378ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = BecomeUpdater
1112025-06-04T01:30:06.389ZDEBGnexus (SagaExecutor): inherited lock from e755593d-8ac7-4b40-9f18-b1abd5702045 to a7bdc6e6-6858-4b38-be73-f70160601be1 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 5abad387-28b5-4eac-817c-81deb58a742d locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = e755593d-8ac7-4b40-9f18-b1abd5702045 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = BecomeUpdater updater_id = a7bdc6e6-6858-4b38-be73-f70160601be1
1122025-06-04T01:30:06.389ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:987 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1132025-06-04T01:30:06.389ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1142025-06-04T01:30:06.389ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("a7bdc6e6-6858-4b38-be73-f70160601be1")}) node_id = 1 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1152025-06-04T01:30:06.392ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1162025-06-04T01:30:06.392ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1172025-06-04T01:30:06.395ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1182025-06-04T01:30:06.395ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1192025-06-04T01:30:06.399ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1202025-06-04T01:30:06.399ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1212025-06-04T01:30:06.402ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1040 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1222025-06-04T01:30:06.404ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d
1232025-06-04T01:30:06.406ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1242025-06-04T01:30:06.406ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1252025-06-04T01:30:06.416ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1262025-06-04T01:30:06.416ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1272025-06-04T01:30:06.420ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
1282025-06-04T01:30:06.420ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1292025-06-04T01:30:06.420ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UpdateNetworkConfig
1302025-06-04T01:30:06.422ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
1312025-06-04T01:30:06.422ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
1322025-06-04T01:30:06.422ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
1332025-06-04T01:30:06.424ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
1342025-06-04T01:30:06.425ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
1352025-06-04T01:30:06.425ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
1362025-06-04T01:30:06.425ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
1372025-06-04T01:30:06.437ZDEBGnexus: notifying dendrite of updates instance_id = Some(5abad387-28b5-4eac-817c-81deb58a742d (instance)) switch = switch0
1382025-06-04T01:30:06.437ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1743, tv_nsec: 530427453 } })
1392025-06-04T01:30:06.437ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1179
1402025-06-04T01:30:06.468ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
1412025-06-04T01:30:06.468ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1422025-06-04T01:30:06.468ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dafe4dc-9e6a-4814-a0f5-1822f2d9ab42", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
1432025-06-04T01:30:06.468ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1442025-06-04T01:30:06.469ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1452025-06-04T01:30:06.500ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1462025-06-04T01:30:06.500ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1472025-06-04T01:30:06.504ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1482025-06-04T01:30:06.504ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1492025-06-04T01:30:06.512ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
1502025-06-04T01:30:06.513ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
1512025-06-04T01:30:06.528ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1115 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-06-04T01:30:06.514169Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 6a575fb0-c195-4d2a-ac26-16b83f49e832, time_modified: Some(2025-06-04T01:30:06.514169Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, time_modified: Some(2025-06-04T01:30:06.514169Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1522025-06-04T01:30:06.528ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1532025-06-04T01:30:06.528ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1542025-06-04T01:30:06.532ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1552025-06-04T01:30:06.532ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1562025-06-04T01:30:06.537ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1150 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1572025-06-04T01:30:06.537ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UnassignOximeterProducer
1582025-06-04T01:30:06.537ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UnassignOximeterProducer
1592025-06-04T01:30:06.545ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 producer_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1602025-06-04T01:30:06.545ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UnassignOximeterProducer
1612025-06-04T01:30:06.545ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = UnassignOximeterProducer
1622025-06-04T01:30:06.581ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1632025-06-04T01:30:06.581ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/5abad387-28b5-4eac-817c-81deb58a742d
1642025-06-04T01:30:06.607ZDEBGnexus (SagaExecutor): client response oximeter-collector = 5abad387-28b5-4eac-817c-81deb58a742d result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/5abad387-28b5-4eac-817c-81deb58a742d", status: 204, headers: {"x-request-id": "2795e395-2c54-43c1-88d9-dfaaddbfc1c6", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} }) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1652025-06-04T01:30:06.607ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 producer_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1662025-06-04T01:30:06.609ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1672025-06-04T01:30:06.609ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1682025-06-04T01:30:06.612ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1692025-06-04T01:30:06.612ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1702025-06-04T01:30:06.616ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 5abad387-28b5-4eac-817c-81deb58a742d lock = UpdaterLock { updater_id: a7bdc6e6-6858-4b38-be73-f70160601be1, locked_gen: Generation(Generation(3)) } new_intent = None new_runtime = InstanceRuntimeState { time_updated: 2025-06-04T01:30:06.323474108Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1712025-06-04T01:30:06.616ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = CommitInstanceUpdates
1722025-06-04T01:30:06.616ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = CommitInstanceUpdates
1732025-06-04T01:30:06.663ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1201 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d new_runtime = InstanceRuntimeState { time_updated: 2025-06-04T01:30:06.323474108Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
1742025-06-04T01:30:06.664ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1752025-06-04T01:30:06.664ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1762025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Signaled
1772025-06-04T01:30:06.664ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1782025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Signaled
1792025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1802025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1812025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1822025-06-04T01:30:06.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1832025-06-04T01:30:06.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1842025-06-04T01:30:06.667ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1852025-06-04T01:30:06.667ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1862025-06-04T01:30:06.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1872025-06-04T01:30:06.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1882025-06-04T01:30:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1892025-06-04T01:30:06.669ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1902025-06-04T01:30:06.670ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1912025-06-04T01:30:06.670ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("5abad387-28b5-4eac-817c-81deb58a742d"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("a8435d87-6398-4e8f-9fda-269bcd96ac6e")}) node_id = 7 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1922025-06-04T01:30:06.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1932025-06-04T01:30:06.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1942025-06-04T01:30:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1952025-06-04T01:30:06.673ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
1962025-06-04T01:30:06.673ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
1972025-06-04T01:30:06.676ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1982025-06-04T01:30:06.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1992025-06-04T01:30:06.677ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2002025-06-04T01:30:06.677ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2012025-06-04T01:30:06.680ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2022025-06-04T01:30:06.680ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2032025-06-04T01:30:06.683ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
2042025-06-04T01:30:06.683ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ReleaseSledResources
2052025-06-04T01:30:06.683ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ReleaseSledResources
2062025-06-04T01:30:06.684ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2072025-06-04T01:30:06.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2082025-06-04T01:30:06.703ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
2092025-06-04T01:30:06.703ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2102025-06-04T01:30:06.703ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "85d8974a-7c12-4401-93c5-73a094fdf07d", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
2112025-06-04T01:30:06.703ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2122025-06-04T01:30:06.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2132025-06-04T01:30:06.705ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2142025-06-04T01:30:06.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2152025-06-04T01:30:06.706ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2162025-06-04T01:30:06.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2172025-06-04T01:30:06.707ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2182025-06-04T01:30:06.707ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2192025-06-04T01:30:06.708ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2202025-06-04T01:30:06.710ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2212025-06-04T01:30:06.710ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2222025-06-04T01:30:06.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2232025-06-04T01:30:06.710ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2242025-06-04T01:30:06.710ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2252025-06-04T01:30:06.714ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d propolis_id = a8435d87-6398-4e8f-9fda-269bcd96ac6e saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
2262025-06-04T01:30:06.714ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = MarkVmmDeleted
2272025-06-04T01:30:06.714ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = MarkVmmDeleted
2282025-06-04T01:30:06.717ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {db46ce9c-8181-4db6-ab84-1aa1392409d3: {10.151.2.175: {1802f0fa-2695-4546-a144-1c18cd9be354}, 10.151.2.176: {1802f0fa-2695-4546-a144-1c18cd9be354}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
2292025-06-04T01:30:06.717ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2302025-06-04T01:30:06.717ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c7f84c8a-400a-4c8c-b387-c047c2450270", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
2312025-06-04T01:30:06.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2322025-06-04T01:30:06.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2332025-06-04T01:30:06.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2342025-06-04T01:30:06.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2352025-06-04T01:30:06.722ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2362025-06-04T01:30:06.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2372025-06-04T01:30:06.742ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2382025-06-04T01:30:06.742ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "476c4623-b72a-4a42-b1ca-d2bd038dc125", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
2392025-06-04T01:30:06.742ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
2402025-06-04T01:30:06.742ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
2412025-06-04T01:30:06.742ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
2422025-06-04T01:30:06.745ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2432025-06-04T01:30:06.745ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2442025-06-04T01:30:06.745ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 81.041605ms iteration = 25 status = {}
2452025-06-04T01:30:06.745ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "84f2b51c-b83e-47de-b59c-6157fe36aea4", "date": "Wed, 04 Jun 2025 01:30:06 GMT"} })
2462025-06-04T01:30:06.746ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 82.556257ms iteration = 25 status = {}
2472025-06-04T01:30:06.752ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2482025-06-04T01:30:06.752ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2492025-06-04T01:30:06.755ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2502025-06-04T01:30:06.755ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2512025-06-04T01:30:06.758ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2522025-06-04T01:30:06.758ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2532025-06-04T01:30:06.762ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2542025-06-04T01:30:06.762ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2552025-06-04T01:30:06.764ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2562025-06-04T01:30:06.765ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2572025-06-04T01:30:06.766ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2582025-06-04T01:30:06.766ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2592025-06-04T01:30:06.767ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2602025-06-04T01:30:06.768ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2612025-06-04T01:30:06.769ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2622025-06-04T01:30:06.772ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, lookup_type: ById(a645d67d-1f6e-4ba4-9148-3bd6c69e26d4) }, key: 6a575fb0-c195-4d2a-ac26-16b83f49e832, lookup_type: ById(6a575fb0-c195-4d2a-ac26-16b83f49e832) }, key: 5abad387-28b5-4eac-817c-81deb58a742d, lookup_type: ById(5abad387-28b5-4eac-817c-81deb58a742d) } result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2632025-06-04T01:30:06.772ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2642025-06-04T01:30:06.772ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_node = ChainSuccessorSaga
2652025-06-04T01:30:06.837ZWARNnexus (SagaExecutor): instance update: failed to start successor saga! error = failed to fetch latest snapshot for instance file = nexus/src/app/sagas/instance_update/mod.rs:1351 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update
2662025-06-04T01:30:06.837ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2672025-06-04T01:30:06.837ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2682025-06-04T01:30:06.837ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Signaled
2692025-06-04T01:30:06.837ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2702025-06-04T01:30:06.837ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2712025-06-04T01:30:06.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2722025-06-04T01:30:06.839ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2732025-06-04T01:30:06.839ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2742025-06-04T01:30:06.839ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2752025-06-04T01:30:06.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2762025-06-04T01:30:06.842ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2772025-06-04T01:30:06.842ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2782025-06-04T01:30:06.842ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2792025-06-04T01:30:06.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2802025-06-04T01:30:06.845ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2812025-06-04T01:30:06.845ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2822025-06-04T01:30:06.851ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2832025-06-04T01:30:06.851ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2842025-06-04T01:30:06.857ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2852025-06-04T01:30:06.857ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15
2862025-06-04T01:30:06.875ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2872025-06-04T01:30:06.875ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2882025-06-04T01:30:06.875ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 38.81807ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2892025-06-04T01:30:06.899ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2902025-06-04T01:30:06.899ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 saga_name = instance-update saga_result = SagaResult { saga_id: c85e41c0-ce30-48b3-b7cc-87a5a3f04d15, saga_log: SagaLog { saga_id: c85e41c0-ce30-48b3-b7cc-87a5a3f04d15, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("a7bdc6e6-6858-4b38-be73-f70160601be1")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("a7bdc6e6-6858-4b38-be73-f70160601be1")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("5abad387-28b5-4eac-817c-81deb58a742d"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("a8435d87-6398-4e8f-9fda-269bcd96ac6e")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("5abad387-28b5-4eac-817c-81deb58a742d"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("a8435d87-6398-4e8f-9fda-269bcd96ac6e")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("a7bdc6e6-6858-4b38-be73-f70160601be1"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("a7bdc6e6-6858-4b38-be73-f70160601be1")}} }) }
2912025-06-04T01:30:06.899ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = c85e41c0-ce30-48b3-b7cc-87a5a3f04d15 instance_id = 5abad387-28b5-4eac-817c-81deb58a742d saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update
2922025-06-04T01:30:06.899ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2932025-06-04T01:30:06.899ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
2942025-06-04T01:30:06.904ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2952025-06-04T01:30:06.904ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
2962025-06-04T01:30:06.908ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2972025-06-04T01:30:06.908ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
2982025-06-04T01:30:06.912ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
2992025-06-04T01:30:06.912ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0
3002025-06-04T01:30:06.958ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update sec_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
3012025-06-04T01:30:06.958ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 094bc3dc-9776-4eda-8c92-7b41333a4ae0 saga_name = start-instance-update saga_result = SagaResult { saga_id: 094bc3dc-9776-4eda-8c92-7b41333a4ae0, saga_log: SagaLog { saga_id: 094bc3dc-9776-4eda-8c92-7b41333a4ae0, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("e755593d-8ac7-4b40-9f18-b1abd5702045")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("e755593d-8ac7-4b40-9f18-b1abd5702045")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("e755593d-8ac7-4b40-9f18-b1abd5702045"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("e755593d-8ac7-4b40-9f18-b1abd5702045")}} }) }
3022025-06-04T01:30:07.328ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f8d5dbc-0a34-489f-a60c-75c120a26578 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3032025-06-04T01:30:07.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f8d5dbc-0a34-489f-a60c-75c120a26578 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3042025-06-04T01:30:07.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f8d5dbc-0a34-489f-a60c-75c120a26578 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3052025-06-04T01:30:07.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
3062025-06-04T01:30:07.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3072025-06-04T01:30:07.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3082025-06-04T01:30:07.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
3092025-06-04T01:30:07.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3102025-06-04T01:30:07.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3112025-06-04T01:30:07.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3122025-06-04T01:30:07.735ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3132025-06-04T01:30:07.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3142025-06-04T01:30:07.737ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.9296ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3152025-06-04T01:30:07.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3162025-06-04T01:30:07.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3172025-06-04T01:30:07.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3182025-06-04T01:30:07.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3192025-06-04T01:30:07.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3202025-06-04T01:30:07.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3212025-06-04T01:30:07.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
3222025-06-04T01:30:07.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3232025-06-04T01:30:07.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3242025-06-04T01:30:07.757ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
3252025-06-04T01:30:07.757ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.292041ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
3262025-06-04T01:30:08.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c318de4d-dd45-4aee-9bf7-45f4b6c5ddfa roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3272025-06-04T01:30:08.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c318de4d-dd45-4aee-9bf7-45f4b6c5ddfa resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3282025-06-04T01:30:08.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c318de4d-dd45-4aee-9bf7-45f4b6c5ddfa response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3292025-06-04T01:30:09.362ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b02091e-9cf3-4980-8096-56d509b9063e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3302025-06-04T01:30:09.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b02091e-9cf3-4980-8096-56d509b9063e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3312025-06-04T01:30:09.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2503 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b02091e-9cf3-4980-8096-56d509b9063e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3322025-06-04T01:30:10.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70e3ac6f-9b57-4f1e-9057-09d3433afaa8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3332025-06-04T01:30:10.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70e3ac6f-9b57-4f1e-9057-09d3433afaa8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3342025-06-04T01:30:10.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2162 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70e3ac6f-9b57-4f1e-9057-09d3433afaa8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3352025-06-04T01:30:10.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 615 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 8dbb52bd-3b02-40e3-9b6e-c6041eab31d0 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
3362025-06-04T01:30:11.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a61fa876-c63f-4b36-9d83-3948eca03ff0 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3372025-06-04T01:30:11.397ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a61fa876-c63f-4b36-9d83-3948eca03ff0 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3382025-06-04T01:30:11.399ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2246 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a61fa876-c63f-4b36-9d83-3948eca03ff0 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3392025-06-04T01:30:12.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cdd68c9d-3d27-498a-82ca-991925d0ce4e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3402025-06-04T01:30:12.412ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cdd68c9d-3d27-498a-82ca-991925d0ce4e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3412025-06-04T01:30:12.414ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cdd68c9d-3d27-498a-82ca-991925d0ce4e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3422025-06-04T01:30:13.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c101ebcc-6766-420f-a72e-fb6ab00e5b3a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3432025-06-04T01:30:13.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c101ebcc-6766-420f-a72e-fb6ab00e5b3a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3442025-06-04T01:30:13.432ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2566 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c101ebcc-6766-420f-a72e-fb6ab00e5b3a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3452025-06-04T01:30:14.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 75c347e6-d03e-4241-b0a4-818eca83ab5e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3462025-06-04T01:30:14.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 75c347e6-d03e-4241-b0a4-818eca83ab5e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3472025-06-04T01:30:14.447ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 75c347e6-d03e-4241-b0a4-818eca83ab5e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3482025-06-04T01:30:15.459ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f757466-3dc9-48ea-97d3-a238f8a3fc29 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3492025-06-04T01:30:15.460ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f757466-3dc9-48ea-97d3-a238f8a3fc29 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3502025-06-04T01:30:15.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5f757466-3dc9-48ea-97d3-a238f8a3fc29 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3512025-06-04T01:30:16.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c5e7aef5-abfa-41e0-8b7c-6cf715df2f7c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3522025-06-04T01:30:16.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c5e7aef5-abfa-41e0-8b7c-6cf715df2f7c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3532025-06-04T01:30:16.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2352 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c5e7aef5-abfa-41e0-8b7c-6cf715df2f7c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3542025-06-04T01:30:17.491ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e5164900-9cdb-4e05-9eba-bb63e52ca6c4 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3552025-06-04T01:30:17.491ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e5164900-9cdb-4e05-9eba-bb63e52ca6c4 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3562025-06-04T01:30:17.493ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2622 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e5164900-9cdb-4e05-9eba-bb63e52ca6c4 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3572025-06-04T01:30:17.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
3582025-06-04T01:30:17.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3592025-06-04T01:30:17.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3602025-06-04T01:30:17.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
3612025-06-04T01:30:17.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3622025-06-04T01:30:17.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3632025-06-04T01:30:17.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3642025-06-04T01:30:17.736ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3652025-06-04T01:30:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3662025-06-04T01:30:17.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3672025-06-04T01:30:17.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3682025-06-04T01:30:17.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3692025-06-04T01:30:17.738ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.64829ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3702025-06-04T01:30:17.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3712025-06-04T01:30:17.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3722025-06-04T01:30:17.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3732025-06-04T01:30:17.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
3742025-06-04T01:30:17.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3752025-06-04T01:30:17.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3762025-06-04T01:30:17.753ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
3772025-06-04T01:30:17.753ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.948768ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
3782025-06-04T01:30:18.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8340dc8d-f203-40da-802b-6cc9c06d3cd2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3792025-06-04T01:30:18.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8340dc8d-f203-40da-802b-6cc9c06d3cd2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3802025-06-04T01:30:18.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2367 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8340dc8d-f203-40da-802b-6cc9c06d3cd2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3812025-06-04T01:30:19.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9ace8059-409f-4aec-b34e-82f5d5d76788 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3822025-06-04T01:30:19.523ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9ace8059-409f-4aec-b34e-82f5d5d76788 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3832025-06-04T01:30:19.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2082 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9ace8059-409f-4aec-b34e-82f5d5d76788 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3842025-06-04T01:30:20.461ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 995 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 0a5e2bcd-a507-4d72-94df-fd6349632693 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
3852025-06-04T01:30:20.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c3023db-bc96-4abc-838c-bc43a58dfd35 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3862025-06-04T01:30:20.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c3023db-bc96-4abc-838c-bc43a58dfd35 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3872025-06-04T01:30:20.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2611 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c3023db-bc96-4abc-838c-bc43a58dfd35 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3882025-06-04T01:30:21.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ffb117-2185-4a09-9e6d-3252cf9363a7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3892025-06-04T01:30:21.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ffb117-2185-4a09-9e6d-3252cf9363a7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3902025-06-04T01:30:21.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2210 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ffb117-2185-4a09-9e6d-3252cf9363a7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3912025-06-04T01:30:22.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00bc4b4f-bded-4986-a894-79faf963134c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3922025-06-04T01:30:22.569ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00bc4b4f-bded-4986-a894-79faf963134c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3932025-06-04T01:30:22.571ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2573 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00bc4b4f-bded-4986-a894-79faf963134c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3942025-06-04T01:30:23.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd0ce3f2-8d04-422b-9cd7-d6f16e992033 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3952025-06-04T01:30:23.585ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd0ce3f2-8d04-422b-9cd7-d6f16e992033 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3962025-06-04T01:30:23.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2868 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd0ce3f2-8d04-422b-9cd7-d6f16e992033 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3972025-06-04T01:30:24.601ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95093ff3-83c2-4136-befa-e3258a1a9620 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3982025-06-04T01:30:24.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95093ff3-83c2-4136-befa-e3258a1a9620 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
3992025-06-04T01:30:24.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2085 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95093ff3-83c2-4136-befa-e3258a1a9620 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4002025-06-04T01:30:25.452ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:56881
4012025-06-04T01:30:25.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 78290023-149c-4aed-a8ad-cfe641a7c5de roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
4022025-06-04T01:30:25.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 78290023-149c-4aed-a8ad-cfe641a7c5de resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
4032025-06-04T01:30:25.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2927 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 78290023-149c-4aed-a8ad-cfe641a7c5de response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
4042025-06-04T01:30:25.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 450d3f08-917a-477b-8a31-3546665dfe87 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
4052025-06-04T01:30:25.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 450d3f08-917a-477b-8a31-3546665dfe87 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
4062025-06-04T01:30:25.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2522 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 450d3f08-917a-477b-8a31-3546665dfe87 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
4072025-06-04T01:30:25.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 943a7e56-18c1-4980-b195-2519ab75272a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4082025-06-04T01:30:25.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 943a7e56-18c1-4980-b195-2519ab75272a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4092025-06-04T01:30:25.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2298 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 943a7e56-18c1-4980-b195-2519ab75272a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4102025-06-04T01:30:26.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 932f15d3-e42f-42f0-aedb-9bb59a6ea2c7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4112025-06-04T01:30:26.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 932f15d3-e42f-42f0-aedb-9bb59a6ea2c7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4122025-06-04T01:30:26.638ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 5043 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 932f15d3-e42f-42f0-aedb-9bb59a6ea2c7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4132025-06-04T01:30:27.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bfd5922-f587-4de6-a78a-b9cb749ba772 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4142025-06-04T01:30:27.652ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bfd5922-f587-4de6-a78a-b9cb749ba772 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4152025-06-04T01:30:27.654ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bfd5922-f587-4de6-a78a-b9cb749ba772 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
4162025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
4172025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
4182025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4192025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4202025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4212025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
4222025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
4232025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
4242025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4252025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
4262025-06-04T01:30:27.732ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:20:27.732304972 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4272025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4282025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
4292025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4302025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4312025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
4322025-06-04T01:30:27.732ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
4332025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.751\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
4342025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
4352025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
4362025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4372025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
4382025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4392025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
4402025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
4412025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4422025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4432025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4442025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4452025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4462025-06-04T01:30:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4472025-06-04T01:30:27.733ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 826079662 } })
4482025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
4492025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4502025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
4512025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
4522025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4532025-06-04T01:30:27.733ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
4542025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
4552025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4562025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
4572025-06-04T01:30:27.733ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4582025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4592025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4602025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
4612025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
4622025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
4632025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4642025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4652025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4662025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4672025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4682025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
4692025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
4702025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4712025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4722025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
4732025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4742025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
4752025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
4762025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4772025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4782025-06-04T01:30:27.733ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 826325919 } })
4792025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4802025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4812025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4822025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4832025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4842025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4852025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4862025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4872025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4882025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4892025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
4902025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.4285ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
4912025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4922025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4932025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4942025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
4952025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.42197ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4962025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4972025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4982025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4992025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5002025-06-04T01:30:27.733ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5012025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5022025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5032025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5042025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
5052025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
5062025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5072025-06-04T01:30:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5082025-06-04T01:30:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5092025-06-04T01:30:27.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5102025-06-04T01:30:27.735ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
5112025-06-04T01:30:27.735ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
5122025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5132025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5142025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
5152025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
5162025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
5172025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5182025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
5192025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
5202025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
5212025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5222025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
5232025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
5242025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
5252025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
5262025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5272025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5282025-06-04T01:30:27.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 829289622 } })
5292025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5302025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5312025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5322025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
5332025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
5342025-06-04T01:30:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5352025-06-04T01:30:27.736ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
5362025-06-04T01:30:27.737ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 830242149 } })
5372025-06-04T01:30:27.737ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:20:27.732304972 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5382025-06-04T01:30:27.737ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.865546ms iteration = 13 status = {"expiration":"2025-06-04T01:20:27.732304972Z","pruned":[]}
5392025-06-04T01:30:27.737ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.717572ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5402025-06-04T01:30:27.737ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
5412025-06-04T01:30:27.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5422025-06-04T01:30:27.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5432025-06-04T01:30:27.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5442025-06-04T01:30:27.738ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5452025-06-04T01:30:27.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5462025-06-04T01:30:27.739ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472025-06-04T01:30:27.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5482025-06-04T01:30:27.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5492025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5502025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5512025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5522025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5532025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5542025-06-04T01:30:27.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5552025-06-04T01:30:27.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5562025-06-04T01:30:27.743ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5572025-06-04T01:30:27.743ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5582025-06-04T01:30:27.743ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5592025-06-04T01:30:27.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5602025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5612025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5622025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5632025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5642025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5652025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5662025-06-04T01:30:27.744ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672025-06-04T01:30:27.745ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5682025-06-04T01:30:27.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5692025-06-04T01:30:27.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5702025-06-04T01:30:27.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5712025-06-04T01:30:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5722025-06-04T01:30:27.746ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5732025-06-04T01:30:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5742025-06-04T01:30:27.747ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752025-06-04T01:30:27.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5762025-06-04T01:30:27.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5772025-06-04T01:30:27.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5782025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5792025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5802025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5812025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5822025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5832025-06-04T01:30:27.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5842025-06-04T01:30:27.750ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5852025-06-04T01:30:27.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5862025-06-04T01:30:27.750ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5872025-06-04T01:30:27.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5882025-06-04T01:30:27.752ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5892025-06-04T01:30:27.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5902025-06-04T01:30:27.752ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5912025-06-04T01:30:27.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5922025-06-04T01:30:27.754ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5932025-06-04T01:30:27.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
5942025-06-04T01:30:27.754ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952025-06-04T01:30:27.754ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 19.026574ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
5962025-06-04T01:30:27.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972025-06-04T01:30:27.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
5982025-06-04T01:30:27.756ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
5992025-06-04T01:30:27.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6002025-06-04T01:30:27.757ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6012025-06-04T01:30:27.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6022025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 23.110858ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
6032025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
6042025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
6052025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
6062025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 23.642974ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
6072025-06-04T01:30:27.759ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.85233ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
6082025-06-04T01:30:27.761ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6092025-06-04T01:30:27.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6102025-06-04T01:30:27.761ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
6112025-06-04T01:30:27.761ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 25.902367ms iteration = 13 status = {"by_rx":{},"error":null}
6122025-06-04T01:30:27.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6132025-06-04T01:30:27.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6142025-06-04T01:30:27.762ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.422416ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6152025-06-04T01:30:27.763ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6162025-06-04T01:30:27.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6172025-06-04T01:30:27.764ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6182025-06-04T01:30:27.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6192025-06-04T01:30:27.765ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1764, tv_nsec: 858872512 } })
6202025-06-04T01:30:27.765ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6212025-06-04T01:30:27.766ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
6222025-06-04T01:30:27.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6232025-06-04T01:30:27.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 33.646374ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6242025-06-04T01:30:27.774ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 41.573987ms iteration = 6 status = {"nsuccess":5}
6252025-06-04T01:30:27.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6262025-06-04T01:30:27.777ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6272025-06-04T01:30:27.777ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6282025-06-04T01:30:27.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6292025-06-04T01:30:27.780ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6302025-06-04T01:30:27.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6312025-06-04T01:30:27.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
6322025-06-04T01:30:27.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6332025-06-04T01:30:27.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6342025-06-04T01:30:27.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6352025-06-04T01:30:27.782ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6362025-06-04T01:30:27.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6372025-06-04T01:30:27.787ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
6382025-06-04T01:30:27.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
6392025-06-04T01:30:27.793ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 57.178865ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
6402025-06-04T01:30:27.804ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6412025-06-04T01:30:27.804ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6422025-06-04T01:30:27.804ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6432025-06-04T01:30:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6442025-06-04T01:30:27.807ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6452025-06-04T01:30:27.807ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6462025-06-04T01:30:27.807ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6472025-06-04T01:30:27.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6482025-06-04T01:30:27.808ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6492025-06-04T01:30:27.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6502025-06-04T01:30:27.809ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
6512025-06-04T01:30:27.809ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.33751ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
6522025-06-04T01:30:27.810ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6532025-06-04T01:30:27.810ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6542025-06-04T01:30:27.810ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 75.745176ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6552025-06-04T01:30:27.812ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6562025-06-04T01:30:27.812ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6572025-06-04T01:30:27.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6582025-06-04T01:30:27.813ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6592025-06-04T01:30:27.813ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6602025-06-04T01:30:27.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6612025-06-04T01:30:27.814ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
6622025-06-04T01:30:27.814ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.858159ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6632025-06-04T01:30:27.814ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6642025-06-04T01:30:27.814ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6652025-06-04T01:30:27.814ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6662025-06-04T01:30:27.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6672025-06-04T01:30:27.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6682025-06-04T01:30:27.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6692025-06-04T01:30:27.815ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 79.627825ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6702025-06-04T01:30:27.817ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712025-06-04T01:30:27.817ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6722025-06-04T01:30:27.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6732025-06-04T01:30:27.819ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6742025-06-04T01:30:27.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6752025-06-04T01:30:27.819ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6762025-06-04T01:30:27.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6772025-06-04T01:30:27.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6782025-06-04T01:30:27.821ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6792025-06-04T01:30:27.821ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 88.391711ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
6802025-06-04T01:30:27.821ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6812025-06-04T01:30:27.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6822025-06-04T01:30:27.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
6832025-06-04T01:30:27.824ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6842025-06-04T01:30:27.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6852025-06-04T01:30:27.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6862025-06-04T01:30:27.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6872025-06-04T01:30:27.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6882025-06-04T01:30:27.826ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.205866ms iteration = 25 status = 0
6892025-06-04T01:30:27.827ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6902025-06-04T01:30:27.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6912025-06-04T01:30:27.828ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6922025-06-04T01:30:27.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6932025-06-04T01:30:27.829ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6942025-06-04T01:30:27.830ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6952025-06-04T01:30:27.831ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 98.720281ms iteration = 28 status = {"physical_disks_added":0}
6962025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6972025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6982025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6992025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7002025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7012025-06-04T01:30:27.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7022025-06-04T01:30:27.834ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
7032025-06-04T01:30:27.834ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 100.846121ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7042025-06-04T01:30:27.843ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7052025-06-04T01:30:27.843ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7062025-06-04T01:30:27.843ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.841884ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7072025-06-04T01:30:27.844ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 108.661761ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7082025-06-04T01:30:27.846ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7092025-06-04T01:30:27.846ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7102025-06-04T01:30:27.846ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7112025-06-04T01:30:27.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7122025-06-04T01:30:27.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7132025-06-04T01:30:27.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142025-06-04T01:30:27.851ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7152025-06-04T01:30:27.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7162025-06-04T01:30:27.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7172025-06-04T01:30:27.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7182025-06-04T01:30:27.854ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
7192025-06-04T01:30:27.854ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7202025-06-04T01:30:27.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7212025-06-04T01:30:27.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7222025-06-04T01:30:27.855ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7232025-06-04T01:30:27.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7242025-06-04T01:30:27.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7252025-06-04T01:30:27.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7262025-06-04T01:30:27.863ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
7272025-06-04T01:30:27.863ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7282025-06-04T01:30:27.863ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
7292025-06-04T01:30:27.863ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 131.44904ms iteration = 13 status = {"generation":1}
7302025-06-04T01:30:27.866ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7312025-06-04T01:30:27.866ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
7322025-06-04T01:30:27.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7332025-06-04T01:30:27.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7342025-06-04T01:30:27.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7352025-06-04T01:30:27.870ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7362025-06-04T01:30:27.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7372025-06-04T01:30:27.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7382025-06-04T01:30:27.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7392025-06-04T01:30:27.875ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
7402025-06-04T01:30:27.875ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7412025-06-04T01:30:27.875ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11b4e305-056d-421f-800e-7ebce0346c8b", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:27 GMT"} })
7422025-06-04T01:30:27.875ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7432025-06-04T01:30:27.878ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7442025-06-04T01:30:27.882ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7452025-06-04T01:30:27.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7462025-06-04T01:30:27.889ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
7472025-06-04T01:30:27.889ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 156.819713ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
7482025-06-04T01:30:27.890ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
7492025-06-04T01:30:27.890ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7502025-06-04T01:30:27.890ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfb0e10a-8a3a-4dbf-b577-b754eb55f8f0", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:27 GMT"} })
7512025-06-04T01:30:27.890ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7522025-06-04T01:30:27.891ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7532025-06-04T01:30:27.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7542025-06-04T01:30:27.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7552025-06-04T01:30:27.989ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
7562025-06-04T01:30:27.989ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7572025-06-04T01:30:27.990ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4467b81f-8a05-4fed-9e3c-b5f126482f55", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:27 GMT"} })
7582025-06-04T01:30:27.990ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7592025-06-04T01:30:27.991ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
7602025-06-04T01:30:27.995ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7612025-06-04T01:30:27.996ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "0ee54f6e-9c96-4e9d-8972-b554c1581eb2", "date": "Wed, 04 Jun 2025 01:30:27 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7622025-06-04T01:30:27.996ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7632025-06-04T01:30:27.997ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "60a17e8c-2757-411e-b811-a8e048c9a8b9", "date": "Wed, 04 Jun 2025 01:30:27 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7642025-06-04T01:30:27.997ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7652025-06-04T01:30:27.998ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7662025-06-04T01:30:27.998ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 266.209333ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7672025-06-04T01:30:28.024ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7682025-06-04T01:30:28.026ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 293.88567ms iteration = 25 status = {}
7692025-06-04T01:30:28.030ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7702025-06-04T01:30:28.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7712025-06-04T01:30:28.035ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7722025-06-04T01:30:28.035ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7732025-06-04T01:30:28.035ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "88ba2198-e061-4757-8879-81112969ce10", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
7742025-06-04T01:30:28.035ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7752025-06-04T01:30:28.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7762025-06-04T01:30:28.037ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
7772025-06-04T01:30:28.037ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7782025-06-04T01:30:28.037ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
7792025-06-04T01:30:28.037ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 305.612819ms iteration = 13 status = {"generation":2}
7802025-06-04T01:30:28.037ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7812025-06-04T01:30:28.038ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 306.317269ms iteration = 25 status = null
7822025-06-04T01:30:28.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7832025-06-04T01:30:28.039ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7842025-06-04T01:30:28.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7852025-06-04T01:30:28.041ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7862025-06-04T01:30:28.043ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7872025-06-04T01:30:28.043ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7882025-06-04T01:30:28.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7892025-06-04T01:30:28.066ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7902025-06-04T01:30:28.067ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b3a9a56a-b5d5-4579-833b-dc00428c14ac", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7912025-06-04T01:30:28.067ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7922025-06-04T01:30:28.069ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cc967ba1-076b-49e9-8b16-0c978ccbdbc5", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7932025-06-04T01:30:28.069ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7942025-06-04T01:30:28.071ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "36092e89-050d-4be1-a0df-7686646acba5", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7952025-06-04T01:30:28.071ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7962025-06-04T01:30:28.073ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
7972025-06-04T01:30:28.073ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 341.206997ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
7982025-06-04T01:30:28.081ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7992025-06-04T01:30:28.081ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8002025-06-04T01:30:28.081ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5555527c-bc5a-49e5-b4c7-5db6252269cc", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
8012025-06-04T01:30:28.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8022025-06-04T01:30:28.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8032025-06-04T01:30:28.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8042025-06-04T01:30:28.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8052025-06-04T01:30:28.086ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8062025-06-04T01:30:28.087ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9afaa8aa-3802-4693-9d5b-121b1d6ba31a", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
8072025-06-04T01:30:28.087ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8082025-06-04T01:30:28.087ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8092025-06-04T01:30:28.087ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8102025-06-04T01:30:28.087ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 354.607954ms iteration = 26 status = {}
8112025-06-04T01:30:28.087ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8122025-06-04T01:30:28.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8132025-06-04T01:30:28.091ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8142025-06-04T01:30:28.091ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8152025-06-04T01:30:28.091ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a06d015c-3a10-496a-b8c8-0521d7c932ae", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
8162025-06-04T01:30:28.093ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 360.611103ms iteration = 26 status = {}
8172025-06-04T01:30:28.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8182025-06-04T01:30:28.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8192025-06-04T01:30:28.126ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
8202025-06-04T01:30:28.126ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8212025-06-04T01:30:28.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8222025-06-04T01:30:28.130ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
8232025-06-04T01:30:28.130ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 397.730725ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
8242025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8252025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
8262025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8272025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8282025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8292025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8302025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8312025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8322025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8332025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8342025-06-04T01:30:28.137ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
8352025-06-04T01:30:28.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8362025-06-04T01:30:28.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8372025-06-04T01:30:28.140ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
8382025-06-04T01:30:28.140ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8392025-06-04T01:30:28.237ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "31711077-61cd-4737-8617-da579e7d89f8", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
8402025-06-04T01:30:28.237ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
8412025-06-04T01:30:28.237ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8422025-06-04T01:30:28.237ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9149536-7bfa-4822-b80d-cb922c13274b", "content-length": "682", "date": "Wed, 04 Jun 2025 01:30:28 GMT"} })
8432025-06-04T01:30:28.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8442025-06-04T01:30:28.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8452025-06-04T01:30:28.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8462025-06-04T01:30:28.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8472025-06-04T01:30:28.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8482025-06-04T01:30:28.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8492025-06-04T01:30:28.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8502025-06-04T01:30:28.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8512025-06-04T01:30:28.255ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 522.571638ms iteration = 25 status = {}
8522025-06-04T01:30:28.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 40688758-b46b-417b-8a52-34c0b6ef9e75 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8532025-06-04T01:30:28.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 40688758-b46b-417b-8a52-34c0b6ef9e75 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8542025-06-04T01:30:28.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2697 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 40688758-b46b-417b-8a52-34c0b6ef9e75 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8552025-06-04T01:30:29.693ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3fee437f-6186-4893-b24e-cf18fc9264c8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8562025-06-04T01:30:29.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3fee437f-6186-4893-b24e-cf18fc9264c8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8572025-06-04T01:30:29.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2986 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3fee437f-6186-4893-b24e-cf18fc9264c8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8582025-06-04T01:30:30.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 642 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = a9cbff26-777e-4bbb-ad09-cee7ad8baea3 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
8592025-06-04T01:30:30.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c677c723-43e0-4309-a3e4-7ab294cfd951 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8602025-06-04T01:30:30.711ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c677c723-43e0-4309-a3e4-7ab294cfd951 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8612025-06-04T01:30:30.713ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c677c723-43e0-4309-a3e4-7ab294cfd951 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8622025-06-04T01:30:31.726ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac154ece-7c11-49fc-88bb-11ec967e296d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8632025-06-04T01:30:31.727ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac154ece-7c11-49fc-88bb-11ec967e296d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8642025-06-04T01:30:31.729ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3358 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac154ece-7c11-49fc-88bb-11ec967e296d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8652025-06-04T01:30:32.743ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0ef6d822-d2d8-4e48-8ea1-f0dca6ceda7d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8662025-06-04T01:30:32.744ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0ef6d822-d2d8-4e48-8ea1-f0dca6ceda7d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8672025-06-04T01:30:32.746ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2816 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0ef6d822-d2d8-4e48-8ea1-f0dca6ceda7d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8682025-06-04T01:30:33.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 85802bf9-710a-4643-8d33-9d3487a09c43 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8692025-06-04T01:30:33.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 85802bf9-710a-4643-8d33-9d3487a09c43 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8702025-06-04T01:30:33.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2595 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 85802bf9-710a-4643-8d33-9d3487a09c43 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8712025-06-04T01:30:34.775ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99a22c7c-8fc2-4e54-b156-f55640f0f468 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8722025-06-04T01:30:34.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99a22c7c-8fc2-4e54-b156-f55640f0f468 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8732025-06-04T01:30:34.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2505 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99a22c7c-8fc2-4e54-b156-f55640f0f468 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8742025-06-04T01:30:35.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bc82fe1-4192-4a4c-a4dd-55f5648d6d98 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8752025-06-04T01:30:35.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bc82fe1-4192-4a4c-a4dd-55f5648d6d98 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8762025-06-04T01:30:35.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2664 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3bc82fe1-4192-4a4c-a4dd-55f5648d6d98 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8772025-06-04T01:30:36.807ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b93712a1-c61b-4f5b-8d81-c49921718127 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8782025-06-04T01:30:36.807ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b93712a1-c61b-4f5b-8d81-c49921718127 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8792025-06-04T01:30:36.809ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2085 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b93712a1-c61b-4f5b-8d81-c49921718127 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
8802025-06-04T01:30:37.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
8812025-06-04T01:30:37.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8822025-06-04T01:30:37.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8832025-06-04T01:30:37.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
8842025-06-04T01:30:37.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8852025-06-04T01:30:37.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8862025-06-04T01:30:37.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8872025-06-04T01:30:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8882025-06-04T01:30:37.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8892025-06-04T01:30:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8902025-06-04T01:30:37.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8912025-06-04T01:30:37.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8922025-06-04T01:30:37.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8932025-06-04T01:30:37.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
8942025-06-04T01:30:37.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8952025-06-04T01:30:37.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8962025-06-04T01:30:37.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
8972025-06-04T01:30:37.758ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.154564ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
8982025-06-04T01:30:37.767ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8992025-06-04T01:30:37.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9002025-06-04T01:30:37.769ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.00067ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9012025-06-04T01:30:37.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 852d6ee6-136e-46b1-b0a5-3a84756edb86 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9022025-06-04T01:30:37.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 852d6ee6-136e-46b1-b0a5-3a84756edb86 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9032025-06-04T01:30:37.824ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1641 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 852d6ee6-136e-46b1-b0a5-3a84756edb86 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9042025-06-04T01:30:38.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 778f7eac-badc-4ac9-8d0d-55de2efd2c9e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9052025-06-04T01:30:38.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 778f7eac-badc-4ac9-8d0d-55de2efd2c9e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9062025-06-04T01:30:38.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2281 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 778f7eac-badc-4ac9-8d0d-55de2efd2c9e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9072025-06-04T01:30:39.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 14f21cb9-bc0f-42e0-864a-a3a3fc302ef3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9082025-06-04T01:30:39.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 14f21cb9-bc0f-42e0-864a-a3a3fc302ef3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9092025-06-04T01:30:39.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2481 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 14f21cb9-bc0f-42e0-864a-a3a3fc302ef3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9102025-06-04T01:30:40.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = cfa11344-7caa-4887-a2ce-08ca3e7140ce roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
9112025-06-04T01:30:40.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = cfa11344-7caa-4887-a2ce-08ca3e7140ce resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
9122025-06-04T01:30:40.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = cfa11344-7caa-4887-a2ce-08ca3e7140ce response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
9132025-06-04T01:30:40.456ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = bbd4b447-1ade-494a-9649-819a12b84735 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
9142025-06-04T01:30:40.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = bbd4b447-1ade-494a-9649-819a12b84735 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
9152025-06-04T01:30:40.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2471 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = bbd4b447-1ade-494a-9649-819a12b84735 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
9162025-06-04T01:30:40.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 525 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = d45b04a1-9c5e-4b4a-b539-23bd8b47425d response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
9172025-06-04T01:30:40.869ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd7c7dd6-cf10-4ec1-8fa3-0dacaff50bb3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9182025-06-04T01:30:40.869ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd7c7dd6-cf10-4ec1-8fa3-0dacaff50bb3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9192025-06-04T01:30:40.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2890 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd7c7dd6-cf10-4ec1-8fa3-0dacaff50bb3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9202025-06-04T01:30:41.885ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = def35fef-3afa-498d-a7ce-3a995ad9c6c1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9212025-06-04T01:30:41.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = def35fef-3afa-498d-a7ce-3a995ad9c6c1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9222025-06-04T01:30:41.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2898 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = def35fef-3afa-498d-a7ce-3a995ad9c6c1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9232025-06-04T01:30:42.901ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02b7d78b-5bd8-43ba-8b4f-f2bdd861a8f5 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9242025-06-04T01:30:42.902ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02b7d78b-5bd8-43ba-8b4f-f2bdd861a8f5 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9252025-06-04T01:30:42.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2858 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02b7d78b-5bd8-43ba-8b4f-f2bdd861a8f5 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9262025-06-04T01:30:43.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 44867cda-a2aa-4dc7-b710-6f6703ba712f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9272025-06-04T01:30:43.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 44867cda-a2aa-4dc7-b710-6f6703ba712f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9282025-06-04T01:30:43.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2774 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 44867cda-a2aa-4dc7-b710-6f6703ba712f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9292025-06-04T01:30:44.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9214287d-f0ad-4137-a2ab-0f6b4d1383db roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9302025-06-04T01:30:44.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9214287d-f0ad-4137-a2ab-0f6b4d1383db resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9312025-06-04T01:30:44.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2272 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9214287d-f0ad-4137-a2ab-0f6b4d1383db response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9322025-06-04T01:30:45.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8e9d0f0d-aab7-481e-8af5-3b7b7db1667a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9332025-06-04T01:30:45.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8e9d0f0d-aab7-481e-8af5-3b7b7db1667a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9342025-06-04T01:30:45.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8e9d0f0d-aab7-481e-8af5-3b7b7db1667a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9352025-06-04T01:30:46.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4503cc88-862b-4702-8e52-571bdca1c274 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9362025-06-04T01:30:46.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4503cc88-862b-4702-8e52-571bdca1c274 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9372025-06-04T01:30:46.968ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2664 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4503cc88-862b-4702-8e52-571bdca1c274 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9382025-06-04T01:30:47.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
9392025-06-04T01:30:47.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9402025-06-04T01:30:47.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9412025-06-04T01:30:47.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
9422025-06-04T01:30:47.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9432025-06-04T01:30:47.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9442025-06-04T01:30:47.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452025-06-04T01:30:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9462025-06-04T01:30:47.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9472025-06-04T01:30:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9482025-06-04T01:30:47.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9492025-06-04T01:30:47.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9502025-06-04T01:30:47.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9512025-06-04T01:30:47.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
9522025-06-04T01:30:47.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9532025-06-04T01:30:47.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9542025-06-04T01:30:47.755ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
9552025-06-04T01:30:47.755ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.831439ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
9562025-06-04T01:30:47.766ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9572025-06-04T01:30:47.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9582025-06-04T01:30:47.768ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.945571ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9592025-06-04T01:30:47.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c214f85-c4d9-412b-b1ea-c1ff95869f7f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9602025-06-04T01:30:47.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c214f85-c4d9-412b-b1ea-c1ff95869f7f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9612025-06-04T01:30:47.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c214f85-c4d9-412b-b1ea-c1ff95869f7f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9622025-06-04T01:30:48.997ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e6d8981d-a5b8-43e2-b545-45497663f5d1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9632025-06-04T01:30:48.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e6d8981d-a5b8-43e2-b545-45497663f5d1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9642025-06-04T01:30:48.999ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e6d8981d-a5b8-43e2-b545-45497663f5d1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9652025-06-04T01:30:50.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1749ac1-3087-413b-900c-bc2b0797aedf roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9662025-06-04T01:30:50.014ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1749ac1-3087-413b-900c-bc2b0797aedf resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9672025-06-04T01:30:50.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1749ac1-3087-413b-900c-bc2b0797aedf response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9682025-06-04T01:30:50.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 747 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 20e25dee-3d66-4564-a416-0a895c001baa response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
9692025-06-04T01:30:51.028ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c43af2a7-8b62-43ca-a3a0-a04c1d828765 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9702025-06-04T01:30:51.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c43af2a7-8b62-43ca-a3a0-a04c1d828765 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9712025-06-04T01:30:51.031ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c43af2a7-8b62-43ca-a3a0-a04c1d828765 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9722025-06-04T01:30:52.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 419a8289-8303-46a2-9aa5-6278588305db roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9732025-06-04T01:30:52.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 419a8289-8303-46a2-9aa5-6278588305db resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9742025-06-04T01:30:52.046ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2361 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 419a8289-8303-46a2-9aa5-6278588305db response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9752025-06-04T01:30:53.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2c39977-2b90-4c46-bd17-c405c1975e37 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9762025-06-04T01:30:53.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2c39977-2b90-4c46-bd17-c405c1975e37 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9772025-06-04T01:30:53.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2220 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2c39977-2b90-4c46-bd17-c405c1975e37 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9782025-06-04T01:30:54.075ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = edcd41f1-f9bf-461e-88c3-ec80e78228d3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9792025-06-04T01:30:54.076ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = edcd41f1-f9bf-461e-88c3-ec80e78228d3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9802025-06-04T01:30:54.078ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2570 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = edcd41f1-f9bf-461e-88c3-ec80e78228d3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9812025-06-04T01:30:55.091ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a726974e-a8dc-47d5-beba-eff70d6fefa8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9822025-06-04T01:30:55.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a726974e-a8dc-47d5-beba-eff70d6fefa8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9832025-06-04T01:30:55.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2271 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a726974e-a8dc-47d5-beba-eff70d6fefa8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9842025-06-04T01:30:56.107ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b134018-7fba-42f7-aa3e-abac08f67580 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9852025-06-04T01:30:56.108ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b134018-7fba-42f7-aa3e-abac08f67580 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9862025-06-04T01:30:56.109ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2077 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2b134018-7fba-42f7-aa3e-abac08f67580 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9872025-06-04T01:30:57.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dcf69ca6-965c-4e79-b8b1-87e455d8b9b2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9882025-06-04T01:30:57.122ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dcf69ca6-965c-4e79-b8b1-87e455d8b9b2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9892025-06-04T01:30:57.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3020 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dcf69ca6-965c-4e79-b8b1-87e455d8b9b2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
9902025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
9912025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
9922025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9932025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9942025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
9952025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
9962025-06-04T01:30:57.732ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9972025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9982025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9992025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10002025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
10012025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
10022025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10032025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
10042025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10052025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10062025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
10072025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10082025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
10092025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10102025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
10112025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
10122025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10132025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10142025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
10152025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10162025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10172025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
10182025-06-04T01:30:57.733ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10192025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10202025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
10212025-06-04T01:30:57.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10222025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10232025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10242025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10252025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10262025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10272025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10282025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10292025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10302025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10312025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10322025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
10332025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
10342025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10352025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
10362025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10372025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10382025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
10392025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
10402025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10412025-06-04T01:30:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10422025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10432025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
10442025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10452025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10462025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10472025-06-04T01:30:57.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10482025-06-04T01:30:57.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1794, tv_nsec: 830004882 } })
10492025-06-04T01:30:57.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1794, tv_nsec: 830312811 } })
10502025-06-04T01:30:57.737ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
10512025-06-04T01:30:57.737ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
10522025-06-04T01:30:57.737ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10532025-06-04T01:30:57.737ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.053902ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10542025-06-04T01:30:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10552025-06-04T01:30:57.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10562025-06-04T01:30:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10572025-06-04T01:30:57.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10582025-06-04T01:30:57.739ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10592025-06-04T01:30:57.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10602025-06-04T01:30:57.740ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10612025-06-04T01:30:57.740ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10632025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10642025-06-04T01:30:57.742ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10652025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
10662025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10672025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10682025-06-04T01:30:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10692025-06-04T01:30:57.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10702025-06-04T01:30:57.743ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10712025-06-04T01:30:57.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10722025-06-04T01:30:57.744ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10732025-06-04T01:30:57.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10742025-06-04T01:30:57.745ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10752025-06-04T01:30:57.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10762025-06-04T01:30:57.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10772025-06-04T01:30:57.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10782025-06-04T01:30:57.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10792025-06-04T01:30:57.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10802025-06-04T01:30:57.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10812025-06-04T01:30:57.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10822025-06-04T01:30:57.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10832025-06-04T01:30:57.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10842025-06-04T01:30:57.750ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10852025-06-04T01:30:57.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10862025-06-04T01:30:57.751ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10872025-06-04T01:30:57.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10882025-06-04T01:30:57.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10892025-06-04T01:30:57.752ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1794, tv_nsec: 845646291 } })
10902025-06-04T01:30:57.752ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
10912025-06-04T01:30:57.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
10922025-06-04T01:30:57.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10932025-06-04T01:30:57.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10942025-06-04T01:30:57.759ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10952025-06-04T01:30:57.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10962025-06-04T01:30:57.761ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10972025-06-04T01:30:57.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10982025-06-04T01:30:57.764ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10992025-06-04T01:30:57.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11002025-06-04T01:30:57.771ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
11012025-06-04T01:30:57.771ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.629344ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
11022025-06-04T01:30:57.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.756423ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
11032025-06-04T01:30:57.786ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11042025-06-04T01:30:57.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11052025-06-04T01:30:57.788ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11062025-06-04T01:30:57.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11072025-06-04T01:30:57.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.621513ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
11082025-06-04T01:30:57.790ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11092025-06-04T01:30:57.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11102025-06-04T01:30:57.792ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 58.926464ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11112025-06-04T01:30:57.793ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 58.77881ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
11122025-06-04T01:30:57.794ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11132025-06-04T01:30:57.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11142025-06-04T01:30:57.794ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11152025-06-04T01:30:57.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11162025-06-04T01:30:57.795ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11172025-06-04T01:30:57.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11182025-06-04T01:30:57.796ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11192025-06-04T01:30:57.796ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 62.688699ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11202025-06-04T01:30:57.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11212025-06-04T01:30:57.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11222025-06-04T01:30:57.797ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 62.774212ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11232025-06-04T01:30:57.798ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11242025-06-04T01:30:57.798ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11252025-06-04T01:30:57.798ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11262025-06-04T01:30:57.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11272025-06-04T01:30:57.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11282025-06-04T01:30:57.798ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11292025-06-04T01:30:57.800ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11302025-06-04T01:30:57.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11312025-06-04T01:30:57.801ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11322025-06-04T01:30:57.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11332025-06-04T01:30:57.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
11342025-06-04T01:30:57.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11352025-06-04T01:30:57.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11362025-06-04T01:30:57.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11372025-06-04T01:30:57.802ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11382025-06-04T01:30:57.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11392025-06-04T01:30:57.803ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11402025-06-04T01:30:57.804ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 70.866239ms iteration = 26 status = 0
11412025-06-04T01:30:57.806ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11422025-06-04T01:30:57.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11432025-06-04T01:30:57.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11442025-06-04T01:30:57.808ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11452025-06-04T01:30:57.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11462025-06-04T01:30:57.812ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
11472025-06-04T01:30:57.812ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11482025-06-04T01:30:57.813ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "71b61092-9670-4181-938f-91709a2b5f27", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11492025-06-04T01:30:57.813ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11502025-06-04T01:30:57.814ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11512025-06-04T01:30:57.816ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
11522025-06-04T01:30:57.816ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11532025-06-04T01:30:57.817ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "907d790d-cb15-45e6-b69f-59f7352a5bf3", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11542025-06-04T01:30:57.817ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11552025-06-04T01:30:57.820ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11562025-06-04T01:30:57.821ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11572025-06-04T01:30:57.823ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11582025-06-04T01:30:57.823ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11592025-06-04T01:30:57.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11602025-06-04T01:30:57.826ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 91.132858ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11612025-06-04T01:30:57.836ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 102.480426ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11622025-06-04T01:30:57.846ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11632025-06-04T01:30:57.846ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11642025-06-04T01:30:57.846ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 112.481567ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
11652025-06-04T01:30:57.849ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662025-06-04T01:30:57.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11672025-06-04T01:30:57.851ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11682025-06-04T01:30:57.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11692025-06-04T01:30:57.854ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
11702025-06-04T01:30:57.854ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11712025-06-04T01:30:57.854ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "388d5542-2e9e-433b-be3f-0bf09997b6aa", "content-length": "26", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11722025-06-04T01:30:57.854ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11732025-06-04T01:30:57.855ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
11742025-06-04T01:30:57.886ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
11752025-06-04T01:30:57.886ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 153.070126ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11762025-06-04T01:30:57.908ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 175.136376ms iteration = 29 status = {"physical_disks_added":0}
11772025-06-04T01:30:57.926ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11782025-06-04T01:30:57.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11792025-06-04T01:30:57.929ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 196.514556ms iteration = 26 status = null
11802025-06-04T01:30:57.997ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11812025-06-04T01:30:57.997ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11822025-06-04T01:30:57.997ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "01fbc32f-5c13-4632-9815-c4180a6ca60a", "content-length": "372", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11832025-06-04T01:30:57.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11842025-06-04T01:30:57.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11852025-06-04T01:30:58.002ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11862025-06-04T01:30:58.002ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "69aa2421-a28d-4dd8-b1b9-13108ec61e8c", "content-length": "691", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11872025-06-04T01:30:58.002ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
11882025-06-04T01:30:58.002ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
11892025-06-04T01:30:58.002ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
11902025-06-04T01:30:58.003ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 269.780803ms iteration = 27 status = {}
11912025-06-04T01:30:58.010ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 277.654244ms iteration = 26 status = {}
11922025-06-04T01:30:58.044ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
11932025-06-04T01:30:58.044ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11942025-06-04T01:30:58.044ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dd35f84f-6607-432d-8dfa-aaeee0475ff5", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
11952025-06-04T01:30:58.044ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11962025-06-04T01:30:58.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11972025-06-04T01:30:58.047ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11982025-06-04T01:30:58.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11992025-06-04T01:30:58.051ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12002025-06-04T01:30:58.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12012025-06-04T01:30:58.054ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
12022025-06-04T01:30:58.054ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12032025-06-04T01:30:58.054ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8642b498-d66b-4e12-b69f-2af54f1ab6c8", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
12042025-06-04T01:30:58.055ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 321.948298ms iteration = 27 status = {}
12052025-06-04T01:30:58.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12062025-06-04T01:30:58.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12072025-06-04T01:30:58.077ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
12082025-06-04T01:30:58.077ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12092025-06-04T01:30:58.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12102025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
12112025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
12122025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
12132025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
12142025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12152025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12162025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12172025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12182025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12192025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12202025-06-04T01:30:58.090ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
12212025-06-04T01:30:58.090ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12222025-06-04T01:30:58.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12232025-06-04T01:30:58.093ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
12242025-06-04T01:30:58.093ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12252025-06-04T01:30:58.189ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7d71903c-8576-4de4-858f-c6f4fd63b940", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
12262025-06-04T01:30:58.189ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
12272025-06-04T01:30:58.189ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12282025-06-04T01:30:58.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c42fd67e-8449-4629-8b30-3507027755bc roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12292025-06-04T01:30:58.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c42fd67e-8449-4629-8b30-3507027755bc resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12302025-06-04T01:30:58.189ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c505323-d92b-4a2f-a59d-94901b2d1ee3", "content-length": "682", "date": "Wed, 04 Jun 2025 01:30:57 GMT"} })
12312025-06-04T01:30:58.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12322025-06-04T01:30:58.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12332025-06-04T01:30:58.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c42fd67e-8449-4629-8b30-3507027755bc response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12342025-06-04T01:30:58.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12352025-06-04T01:30:58.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12362025-06-04T01:30:58.196ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12372025-06-04T01:30:58.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12382025-06-04T01:30:58.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12392025-06-04T01:30:58.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12402025-06-04T01:30:58.205ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 471.160276ms iteration = 26 status = {}
12412025-06-04T01:30:59.205ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f4a6a8c1-85b1-4c0c-abab-b806ef38d357 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12422025-06-04T01:30:59.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f4a6a8c1-85b1-4c0c-abab-b806ef38d357 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12432025-06-04T01:30:59.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2434 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f4a6a8c1-85b1-4c0c-abab-b806ef38d357 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12442025-06-04T01:31:00.223ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02545931-65c6-4cc7-8181-883c525a6b50 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12452025-06-04T01:31:00.223ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02545931-65c6-4cc7-8181-883c525a6b50 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12462025-06-04T01:31:00.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1924 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 02545931-65c6-4cc7-8181-883c525a6b50 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12472025-06-04T01:31:00.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 561 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = be251466-8adc-4ffc-8fc9-d66fe5e75aba response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
12482025-06-04T01:31:01.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ad57bdf3-2525-4101-974a-3ce5824541c2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12492025-06-04T01:31:01.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ad57bdf3-2525-4101-974a-3ce5824541c2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12502025-06-04T01:31:01.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2742 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ad57bdf3-2525-4101-974a-3ce5824541c2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12512025-06-04T01:31:02.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = de0e5e43-9761-419f-b5f8-446bcb5b68ab roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12522025-06-04T01:31:02.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = de0e5e43-9761-419f-b5f8-446bcb5b68ab resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12532025-06-04T01:31:02.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = de0e5e43-9761-419f-b5f8-446bcb5b68ab response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12542025-06-04T01:31:03.271ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bd0123f9-ad0c-4fcd-8fce-fccb8af5b5b7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12552025-06-04T01:31:03.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bd0123f9-ad0c-4fcd-8fce-fccb8af5b5b7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12562025-06-04T01:31:03.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2605 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bd0123f9-ad0c-4fcd-8fce-fccb8af5b5b7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12572025-06-04T01:31:04.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1823d49-2778-4fe0-a132-988052812313 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12582025-06-04T01:31:04.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1823d49-2778-4fe0-a132-988052812313 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12592025-06-04T01:31:04.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2735 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1823d49-2778-4fe0-a132-988052812313 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12602025-06-04T01:31:05.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6715b080-caa7-4616-a57a-44dccb91b2bd roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12612025-06-04T01:31:05.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6715b080-caa7-4616-a57a-44dccb91b2bd resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12622025-06-04T01:31:05.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2141 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6715b080-caa7-4616-a57a-44dccb91b2bd response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12632025-06-04T01:31:06.319ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6cd6496f-b1cd-4650-a8b7-5f780e79665f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12642025-06-04T01:31:06.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6cd6496f-b1cd-4650-a8b7-5f780e79665f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12652025-06-04T01:31:06.322ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2590 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6cd6496f-b1cd-4650-a8b7-5f780e79665f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12662025-06-04T01:31:07.336ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f5324082-5aac-40b0-989a-1c63768417c0 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12672025-06-04T01:31:07.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f5324082-5aac-40b0-989a-1c63768417c0 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12682025-06-04T01:31:07.343ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 7046 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f5324082-5aac-40b0-989a-1c63768417c0 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12692025-06-04T01:31:07.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
12702025-06-04T01:31:07.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12712025-06-04T01:31:07.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12722025-06-04T01:31:07.733ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
12732025-06-04T01:31:07.733ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12742025-06-04T01:31:07.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12752025-06-04T01:31:07.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-06-04T01:31:07.736ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12772025-06-04T01:31:07.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12782025-06-04T01:31:07.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12792025-06-04T01:31:07.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12802025-06-04T01:31:07.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12812025-06-04T01:31:07.738ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.982489ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12822025-06-04T01:31:07.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12832025-06-04T01:31:07.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12842025-06-04T01:31:07.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12852025-06-04T01:31:07.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
12862025-06-04T01:31:07.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12872025-06-04T01:31:07.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12882025-06-04T01:31:07.756ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
12892025-06-04T01:31:07.756ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.261021ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
12902025-06-04T01:31:08.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0629c8c4-c0e3-43a0-9fd2-59c2f53a2cef roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12912025-06-04T01:31:08.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0629c8c4-c0e3-43a0-9fd2-59c2f53a2cef resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12922025-06-04T01:31:08.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0629c8c4-c0e3-43a0-9fd2-59c2f53a2cef response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12932025-06-04T01:31:09.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 31c5bc1f-6422-450c-b0e4-f7cfeea3a20b roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12942025-06-04T01:31:09.373ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 31c5bc1f-6422-450c-b0e4-f7cfeea3a20b resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12952025-06-04T01:31:09.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1931 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 31c5bc1f-6422-450c-b0e4-f7cfeea3a20b response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12962025-06-04T01:31:10.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d65798ad-d8ea-40d7-82be-41328e5e2929 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12972025-06-04T01:31:10.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d65798ad-d8ea-40d7-82be-41328e5e2929 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12982025-06-04T01:31:10.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d65798ad-d8ea-40d7-82be-41328e5e2929 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
12992025-06-04T01:31:10.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 799b5ef9-c12f-487b-b4b2-b04636f595d6 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
13002025-06-04T01:31:10.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 799b5ef9-c12f-487b-b4b2-b04636f595d6 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
13012025-06-04T01:31:10.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2799 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 799b5ef9-c12f-487b-b4b2-b04636f595d6 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
13022025-06-04T01:31:10.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 8701818c-71c9-43b3-963c-a54522b29639 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
13032025-06-04T01:31:10.456ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 8701818c-71c9-43b3-963c-a54522b29639 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
13042025-06-04T01:31:10.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2197 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56881 req_id = 8701818c-71c9-43b3-963c-a54522b29639 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
13052025-06-04T01:31:10.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 509 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = acba519f-5ffd-4fca-ae93-54640e96d5e0 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
13062025-06-04T01:31:11.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b0d8cb62-b651-4617-abd8-c96d4f2c56f9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13072025-06-04T01:31:11.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b0d8cb62-b651-4617-abd8-c96d4f2c56f9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13082025-06-04T01:31:11.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2611 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b0d8cb62-b651-4617-abd8-c96d4f2c56f9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13092025-06-04T01:31:12.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 57dcde70-7d16-41e4-a45a-5e7c861625f3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13102025-06-04T01:31:12.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 57dcde70-7d16-41e4-a45a-5e7c861625f3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13112025-06-04T01:31:12.421ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 57dcde70-7d16-41e4-a45a-5e7c861625f3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13122025-06-04T01:31:13.434ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f6d299e1-95e6-44d9-aa23-ba47c00236fc roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13132025-06-04T01:31:13.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f6d299e1-95e6-44d9-aa23-ba47c00236fc resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13142025-06-04T01:31:13.436ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2328 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f6d299e1-95e6-44d9-aa23-ba47c00236fc response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13152025-06-04T01:31:14.449ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5aa63506-5c3b-4da5-854b-5998e5870713 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13162025-06-04T01:31:14.449ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5aa63506-5c3b-4da5-854b-5998e5870713 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13172025-06-04T01:31:14.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2706 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5aa63506-5c3b-4da5-854b-5998e5870713 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13182025-06-04T01:31:15.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ba8a4a67-0fb1-4452-9807-40d035ae7232 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13192025-06-04T01:31:15.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ba8a4a67-0fb1-4452-9807-40d035ae7232 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13202025-06-04T01:31:15.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2531 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ba8a4a67-0fb1-4452-9807-40d035ae7232 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13212025-06-04T01:31:16.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1afe835a-1234-4243-9c2f-b219e9be4e95 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13222025-06-04T01:31:16.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1afe835a-1234-4243-9c2f-b219e9be4e95 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13232025-06-04T01:31:16.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2265 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1afe835a-1234-4243-9c2f-b219e9be4e95 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13242025-06-04T01:31:17.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1791600c-1fec-480f-8431-d1f391e6d370 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13252025-06-04T01:31:17.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1791600c-1fec-480f-8431-d1f391e6d370 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13262025-06-04T01:31:17.498ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2546 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1791600c-1fec-480f-8431-d1f391e6d370 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13272025-06-04T01:31:17.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
13282025-06-04T01:31:17.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13292025-06-04T01:31:17.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13302025-06-04T01:31:17.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
13312025-06-04T01:31:17.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13322025-06-04T01:31:17.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13332025-06-04T01:31:17.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13342025-06-04T01:31:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13352025-06-04T01:31:17.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13362025-06-04T01:31:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13372025-06-04T01:31:17.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13382025-06-04T01:31:17.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13392025-06-04T01:31:17.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13402025-06-04T01:31:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
13412025-06-04T01:31:17.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13422025-06-04T01:31:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13432025-06-04T01:31:17.755ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
13442025-06-04T01:31:17.755ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.501427ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
13452025-06-04T01:31:17.764ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13462025-06-04T01:31:17.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13472025-06-04T01:31:17.803ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.307143ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13482025-06-04T01:31:18.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97ce28fd-8a6e-4b51-9e68-b457d3d732e7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13492025-06-04T01:31:18.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97ce28fd-8a6e-4b51-9e68-b457d3d732e7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13502025-06-04T01:31:18.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97ce28fd-8a6e-4b51-9e68-b457d3d732e7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13512025-06-04T01:31:19.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0221285-0a48-49f3-a4cd-ea0b9fc97f3a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13522025-06-04T01:31:19.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0221285-0a48-49f3-a4cd-ea0b9fc97f3a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13532025-06-04T01:31:19.530ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0221285-0a48-49f3-a4cd-ea0b9fc97f3a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13542025-06-04T01:31:20.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 798 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 18431cee-2f82-4941-aad8-1d977b04035c response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
13552025-06-04T01:31:20.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e03067a3-839c-4fea-8038-7f9f658d910d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13562025-06-04T01:31:20.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e03067a3-839c-4fea-8038-7f9f658d910d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13572025-06-04T01:31:20.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e03067a3-839c-4fea-8038-7f9f658d910d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13582025-06-04T01:31:21.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1be7c00a-d189-490d-8b72-30179221921e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13592025-06-04T01:31:21.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1be7c00a-d189-490d-8b72-30179221921e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13602025-06-04T01:31:21.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1be7c00a-d189-490d-8b72-30179221921e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13612025-06-04T01:31:22.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c67f013-961a-456e-901e-a44f98950fd3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13622025-06-04T01:31:22.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c67f013-961a-456e-901e-a44f98950fd3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13632025-06-04T01:31:22.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c67f013-961a-456e-901e-a44f98950fd3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13642025-06-04T01:31:23.591ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 825d4f71-6bbb-401c-bc49-0913df98c71b roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13652025-06-04T01:31:23.591ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 825d4f71-6bbb-401c-bc49-0913df98c71b resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13662025-06-04T01:31:23.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2778 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 825d4f71-6bbb-401c-bc49-0913df98c71b response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13672025-06-04T01:31:24.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6c8cefb4-fbbf-45d7-8a09-45352af3731b roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13682025-06-04T01:31:24.609ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6c8cefb4-fbbf-45d7-8a09-45352af3731b resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13692025-06-04T01:31:24.610ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2351 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6c8cefb4-fbbf-45d7-8a09-45352af3731b response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13702025-06-04T01:31:25.624ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9c32d60b-f380-434e-9341-1d50869ad95f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13712025-06-04T01:31:25.624ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9c32d60b-f380-434e-9341-1d50869ad95f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13722025-06-04T01:31:25.626ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2127 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9c32d60b-f380-434e-9341-1d50869ad95f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13732025-06-04T01:31:26.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 770d26d8-b2e3-4729-911f-d6df6a345065 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13742025-06-04T01:31:26.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 770d26d8-b2e3-4729-911f-d6df6a345065 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13752025-06-04T01:31:26.641ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 770d26d8-b2e3-4729-911f-d6df6a345065 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13762025-06-04T01:31:27.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ff82950e-2179-4c21-86bc-a787fcc9ff30 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13772025-06-04T01:31:27.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ff82950e-2179-4c21-86bc-a787fcc9ff30 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13782025-06-04T01:31:27.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2290 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ff82950e-2179-4c21-86bc-a787fcc9ff30 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
13792025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
13802025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
13812025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
13822025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13832025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13842025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
13852025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13862025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
13872025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13882025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
13892025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
13902025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13912025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13922025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
13932025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
13942025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
13952025-06-04T01:31:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
13962025-06-04T01:31:27.732ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
13972025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
13982025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 28.22\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
13992025-06-04T01:31:27.732ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:21:27.732007893 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
14002025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
14012025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
14022025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14032025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14042025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14052025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14062025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14072025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14082025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14092025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
14102025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
14112025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
14122025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14132025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14142025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
14152025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14162025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
14172025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
14182025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14192025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
14202025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14212025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
14222025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14232025-06-04T01:31:27.732ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
14242025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
14252025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
14262025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14272025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
14282025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
14292025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
14302025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
14312025-06-04T01:31:27.732ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
14322025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14332025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
14342025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14352025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
14362025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14372025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14382025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14392025-06-04T01:31:27.732ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1824, tv_nsec: 826532274 } })
14402025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14412025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14422025-06-04T01:31:27.732ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1824, tv_nsec: 826594836 } })
14432025-06-04T01:31:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
14442025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14452025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14462025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14472025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14482025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14492025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14502025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14512025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
14522025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.308197ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
14532025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
14542025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
14552025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
14562025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
14572025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.578414ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
14582025-06-04T01:31:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
14592025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14602025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
14612025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
14622025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
14632025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14642025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
14652025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
14662025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
14672025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14682025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
14692025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
14702025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
14712025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
14722025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
14732025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
14742025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
14752025-06-04T01:31:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
14762025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14772025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
14782025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
14792025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14802025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
14812025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
14822025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
14832025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
14842025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14852025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14862025-06-04T01:31:27.735ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1824, tv_nsec: 829363944 } })
14872025-06-04T01:31:27.735ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14882025-06-04T01:31:27.735ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
14892025-06-04T01:31:27.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1824, tv_nsec: 829766765 } })
14902025-06-04T01:31:27.736ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.27282ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14912025-06-04T01:31:27.736ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
14922025-06-04T01:31:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14932025-06-04T01:31:27.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14942025-06-04T01:31:27.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14952025-06-04T01:31:27.737ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14962025-06-04T01:31:27.737ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14972025-06-04T01:31:27.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14982025-06-04T01:31:27.738ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14992025-06-04T01:31:27.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15002025-06-04T01:31:27.738ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:21:27.732007893 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
15012025-06-04T01:31:27.738ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.934185ms iteration = 14 status = {"expiration":"2025-06-04T01:21:27.732007893Z","pruned":[]}
15022025-06-04T01:31:27.740ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15032025-06-04T01:31:27.740ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15042025-06-04T01:31:27.740ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15052025-06-04T01:31:27.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15062025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15072025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15082025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
15092025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
15102025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15112025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15122025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15132025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
15142025-06-04T01:31:27.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15152025-06-04T01:31:27.743ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15162025-06-04T01:31:27.743ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15172025-06-04T01:31:27.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15182025-06-04T01:31:27.744ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15192025-06-04T01:31:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15202025-06-04T01:31:27.746ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15212025-06-04T01:31:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15222025-06-04T01:31:27.747ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15232025-06-04T01:31:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15242025-06-04T01:31:27.750ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
15252025-06-04T01:31:27.750ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
15262025-06-04T01:31:27.750ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 16.612686ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
15272025-06-04T01:31:27.751ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 16.614077ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
15282025-06-04T01:31:27.752ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
15292025-06-04T01:31:27.752ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 17.783649ms iteration = 14 status = {"by_rx":{},"error":null}
15302025-06-04T01:31:27.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15312025-06-04T01:31:27.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15322025-06-04T01:31:27.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15332025-06-04T01:31:27.753ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15342025-06-04T01:31:27.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15352025-06-04T01:31:27.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15362025-06-04T01:31:27.763ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15372025-06-04T01:31:27.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15382025-06-04T01:31:27.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15392025-06-04T01:31:27.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15402025-06-04T01:31:27.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15412025-06-04T01:31:27.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15422025-06-04T01:31:27.773ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15432025-06-04T01:31:27.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15442025-06-04T01:31:27.782ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15452025-06-04T01:31:27.782ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15462025-06-04T01:31:27.782ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15472025-06-04T01:31:27.782ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15482025-06-04T01:31:27.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15492025-06-04T01:31:27.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15502025-06-04T01:31:27.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 56.218058ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15512025-06-04T01:31:27.790ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 55.226321ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
15522025-06-04T01:31:27.792ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15532025-06-04T01:31:27.793ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 58.283596ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
15542025-06-04T01:31:27.793ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
15552025-06-04T01:31:27.793ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15562025-06-04T01:31:27.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
15572025-06-04T01:31:27.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582025-06-04T01:31:27.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15592025-06-04T01:31:27.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15602025-06-04T01:31:27.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15612025-06-04T01:31:27.796ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.773494ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
15622025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15632025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15642025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15652025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15662025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15672025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15682025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15692025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15702025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15712025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
15722025-06-04T01:31:27.797ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
15732025-06-04T01:31:27.797ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 65.413497ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
15742025-06-04T01:31:27.798ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15752025-06-04T01:31:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15762025-06-04T01:31:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15772025-06-04T01:31:27.798ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15782025-06-04T01:31:27.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15792025-06-04T01:31:27.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15802025-06-04T01:31:27.799ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15812025-06-04T01:31:27.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15822025-06-04T01:31:27.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15832025-06-04T01:31:27.800ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15842025-06-04T01:31:27.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15852025-06-04T01:31:27.801ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 66.731173ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
15862025-06-04T01:31:27.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15872025-06-04T01:31:27.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15882025-06-04T01:31:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
15892025-06-04T01:31:27.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15902025-06-04T01:31:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15912025-06-04T01:31:27.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15922025-06-04T01:31:27.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15932025-06-04T01:31:27.806ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.274485ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15942025-06-04T01:31:27.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15952025-06-04T01:31:27.807ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 72.503816ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15962025-06-04T01:31:27.808ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15972025-06-04T01:31:27.808ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1824, tv_nsec: 902462206 } })
15982025-06-04T01:31:27.808ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
15992025-06-04T01:31:27.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16002025-06-04T01:31:27.809ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16012025-06-04T01:31:27.809ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.748494ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16022025-06-04T01:31:27.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16032025-06-04T01:31:27.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16042025-06-04T01:31:27.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16052025-06-04T01:31:27.813ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
16062025-06-04T01:31:27.813ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
16072025-06-04T01:31:27.813ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16082025-06-04T01:31:27.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16092025-06-04T01:31:27.830ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
16102025-06-04T01:31:27.830ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 99.106252ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
16112025-06-04T01:31:27.842ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16122025-06-04T01:31:27.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16132025-06-04T01:31:27.844ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
16142025-06-04T01:31:27.844ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
16152025-06-04T01:31:27.844ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
16162025-06-04T01:31:27.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16172025-06-04T01:31:27.844ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16182025-06-04T01:31:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16192025-06-04T01:31:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16202025-06-04T01:31:27.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16212025-06-04T01:31:27.847ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16222025-06-04T01:31:27.847ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 112.949691ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16232025-06-04T01:31:27.849ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
16242025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16252025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16262025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
16272025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16282025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16292025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16302025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16312025-06-04T01:31:27.850ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16322025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16332025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16342025-06-04T01:31:27.850ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16352025-06-04T01:31:27.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16362025-06-04T01:31:27.852ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.908496ms iteration = 27 status = 0
16372025-06-04T01:31:27.860ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
16382025-06-04T01:31:27.860ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16392025-06-04T01:31:27.861ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5c37470-459b-46da-9bd5-958a26e1314e", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:27 GMT"} })
16402025-06-04T01:31:27.861ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16412025-06-04T01:31:27.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16422025-06-04T01:31:27.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16432025-06-04T01:31:27.862ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16442025-06-04T01:31:27.866ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16452025-06-04T01:31:27.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16462025-06-04T01:31:27.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16472025-06-04T01:31:27.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16482025-06-04T01:31:27.869ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
16492025-06-04T01:31:27.869ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 137.056997ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16502025-06-04T01:31:27.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16512025-06-04T01:31:27.872ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
16522025-06-04T01:31:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16532025-06-04T01:31:27.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16542025-06-04T01:31:27.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16552025-06-04T01:31:27.878ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
16562025-06-04T01:31:27.878ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
16572025-06-04T01:31:27.878ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
16582025-06-04T01:31:27.878ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 146.08426ms iteration = 14 status = {"generation":1}
16592025-06-04T01:31:27.886ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
16602025-06-04T01:31:27.886ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16612025-06-04T01:31:27.886ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc497dc-5c9e-4dc3-a06a-0b41d735d446", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:27 GMT"} })
16622025-06-04T01:31:27.886ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16632025-06-04T01:31:27.888ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16642025-06-04T01:31:27.888ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16652025-06-04T01:31:27.888ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16662025-06-04T01:31:27.888ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 156.128122ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16672025-06-04T01:31:27.891ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16682025-06-04T01:31:27.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
16692025-06-04T01:31:27.894ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16702025-06-04T01:31:27.894ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
16712025-06-04T01:31:27.894ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16722025-06-04T01:31:27.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16732025-06-04T01:31:27.896ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16742025-06-04T01:31:27.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16752025-06-04T01:31:27.898ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16762025-06-04T01:31:27.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16772025-06-04T01:31:27.901ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
16782025-06-04T01:31:27.901ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
16792025-06-04T01:31:27.901ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
16802025-06-04T01:31:27.901ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 169.172989ms iteration = 14 status = {"generation":2}
16812025-06-04T01:31:27.902ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822025-06-04T01:31:27.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16832025-06-04T01:31:27.903ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16842025-06-04T01:31:27.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16852025-06-04T01:31:27.938ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16862025-06-04T01:31:27.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16872025-06-04T01:31:27.941ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16882025-06-04T01:31:27.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16892025-06-04T01:31:27.942ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16902025-06-04T01:31:27.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16912025-06-04T01:31:27.944ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922025-06-04T01:31:27.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16932025-06-04T01:31:27.946ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16942025-06-04T01:31:27.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16952025-06-04T01:31:27.947ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 215.009955ms iteration = 30 status = {"physical_disks_added":0}
16962025-06-04T01:31:27.953ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
16972025-06-04T01:31:27.953ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 221.2152ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
16982025-06-04T01:31:28.017ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16992025-06-04T01:31:28.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17002025-06-04T01:31:28.019ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
17012025-06-04T01:31:28.019ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 287.099849ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
17022025-06-04T01:31:28.020ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 288.20485ms iteration = 27 status = null
17032025-06-04T01:31:28.022ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
17042025-06-04T01:31:28.022ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1b99f893-0bca-4d54-abfc-02b466ca77fc", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17052025-06-04T01:31:28.022ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17062025-06-04T01:31:28.024ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ebb16475-d3a1-4041-992a-34907def2d15", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17072025-06-04T01:31:28.024ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17082025-06-04T01:31:28.026ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
17092025-06-04T01:31:28.026ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 294.535518ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
17102025-06-04T01:31:28.033ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
17112025-06-04T01:31:28.033ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17122025-06-04T01:31:28.034ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9db5c10-b55e-4b52-a843-fa4559e72493", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:27 GMT"} })
17132025-06-04T01:31:28.034ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17142025-06-04T01:31:28.036ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
17152025-06-04T01:31:28.045ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
17162025-06-04T01:31:28.058ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
17172025-06-04T01:31:28.058ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17182025-06-04T01:31:28.058ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "efab8d32-c6c6-4f78-a034-689e16952219", "content-length": "372", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17192025-06-04T01:31:28.058ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17202025-06-04T01:31:28.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17212025-06-04T01:31:28.094ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17222025-06-04T01:31:28.094ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ced08e-f7dc-47d5-be71-8fdbfc5c9b0d", "content-length": "691", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17232025-06-04T01:31:28.094ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17242025-06-04T01:31:28.094ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17252025-06-04T01:31:28.094ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17262025-06-04T01:31:28.095ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 362.935778ms iteration = 28 status = {}
17272025-06-04T01:31:28.099ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.962309ms iteration = 27 status = {}
17282025-06-04T01:31:28.104ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
17292025-06-04T01:31:28.104ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17302025-06-04T01:31:28.104ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "361f2ebd-bfee-4665-991e-6d9cf4c4f87a", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17312025-06-04T01:31:28.104ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17322025-06-04T01:31:28.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17332025-06-04T01:31:28.107ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17342025-06-04T01:31:28.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17352025-06-04T01:31:28.110ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17362025-06-04T01:31:28.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17372025-06-04T01:31:28.114ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17382025-06-04T01:31:28.114ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17392025-06-04T01:31:28.114ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "713d6908-1d36-457c-94f1-ea3f170c4e1b", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17402025-06-04T01:31:28.116ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 383.945717ms iteration = 28 status = {}
17412025-06-04T01:31:28.116ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
17422025-06-04T01:31:28.117ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4b2319af-4e76-43ed-b3dc-627f6dd16df0", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17432025-06-04T01:31:28.117ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17442025-06-04T01:31:28.118ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "db690c34-2d17-4a84-b48b-4d8d9a3d1f71", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17452025-06-04T01:31:28.118ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17462025-06-04T01:31:28.120ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "eddcdcad-ffe3-4e48-8d28-c7ce54b8b5ab", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17472025-06-04T01:31:28.120ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17482025-06-04T01:31:28.122ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
17492025-06-04T01:31:28.122ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 390.943014ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
17502025-06-04T01:31:28.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17512025-06-04T01:31:28.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17522025-06-04T01:31:28.175ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
17532025-06-04T01:31:28.175ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17542025-06-04T01:31:28.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17552025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17562025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
17572025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17582025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17592025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17602025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17612025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17622025-06-04T01:31:28.187ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17632025-06-04T01:31:28.188ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17642025-06-04T01:31:28.188ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17652025-06-04T01:31:28.188ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
17662025-06-04T01:31:28.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17672025-06-04T01:31:28.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17682025-06-04T01:31:28.190ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
17692025-06-04T01:31:28.190ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17702025-06-04T01:31:28.287ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9e085724-4857-497c-b4bd-e1fec0b9c554", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17712025-06-04T01:31:28.287ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
17722025-06-04T01:31:28.287ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17732025-06-04T01:31:28.287ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e45ad96b-f2f6-4a44-9627-623a5896cf3f", "content-length": "682", "date": "Wed, 04 Jun 2025 01:31:28 GMT"} })
17742025-06-04T01:31:28.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17752025-06-04T01:31:28.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17762025-06-04T01:31:28.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17772025-06-04T01:31:28.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17782025-06-04T01:31:28.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17792025-06-04T01:31:28.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17802025-06-04T01:31:28.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17812025-06-04T01:31:28.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17822025-06-04T01:31:28.303ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 571.10089ms iteration = 27 status = {}
17832025-06-04T01:31:28.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 29b2538d-ba7c-462f-8c33-b58b98450556 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17842025-06-04T01:31:28.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 29b2538d-ba7c-462f-8c33-b58b98450556 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17852025-06-04T01:31:28.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1988 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 29b2538d-ba7c-462f-8c33-b58b98450556 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17862025-06-04T01:31:29.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ca354d74-227d-40e2-bc01-19a4108df8f6 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17872025-06-04T01:31:29.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ca354d74-227d-40e2-bc01-19a4108df8f6 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17882025-06-04T01:31:29.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ca354d74-227d-40e2-bc01-19a4108df8f6 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17892025-06-04T01:31:30.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 580 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 71c9163d-dc04-457d-8794-ee32131792c2 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
17902025-06-04T01:31:30.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 68c335eb-35de-42f0-8d09-64990ee2130f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17912025-06-04T01:31:30.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 68c335eb-35de-42f0-8d09-64990ee2130f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17922025-06-04T01:31:30.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2117 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 68c335eb-35de-42f0-8d09-64990ee2130f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17932025-06-04T01:31:31.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d49aeb7b-7543-4b69-8bb7-6365bede7ba7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17942025-06-04T01:31:31.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d49aeb7b-7543-4b69-8bb7-6365bede7ba7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17952025-06-04T01:31:31.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2535 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d49aeb7b-7543-4b69-8bb7-6365bede7ba7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17962025-06-04T01:31:32.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bf6e7ba1-5fb6-4598-a844-f23ec124ad12 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17972025-06-04T01:31:32.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bf6e7ba1-5fb6-4598-a844-f23ec124ad12 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17982025-06-04T01:31:32.735ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bf6e7ba1-5fb6-4598-a844-f23ec124ad12 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
17992025-06-04T01:31:33.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eece0ef4-7fb0-4a6a-95d7-c3cd6d00de64 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18002025-06-04T01:31:33.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eece0ef4-7fb0-4a6a-95d7-c3cd6d00de64 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18012025-06-04T01:31:33.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eece0ef4-7fb0-4a6a-95d7-c3cd6d00de64 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18022025-06-04T01:31:34.765ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ddb2c26d-23d4-4b96-af24-658f236a7fb2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18032025-06-04T01:31:34.765ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ddb2c26d-23d4-4b96-af24-658f236a7fb2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18042025-06-04T01:31:34.767ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2689 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ddb2c26d-23d4-4b96-af24-658f236a7fb2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18052025-06-04T01:31:35.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 17515234-5ae9-4e9e-8ac7-db8a37894c3d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18062025-06-04T01:31:35.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 17515234-5ae9-4e9e-8ac7-db8a37894c3d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18072025-06-04T01:31:35.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 17515234-5ae9-4e9e-8ac7-db8a37894c3d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18082025-06-04T01:31:36.796ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 302669b0-e7dd-49b8-8586-d2bc02e1fc04 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18092025-06-04T01:31:36.797ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 302669b0-e7dd-49b8-8586-d2bc02e1fc04 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18102025-06-04T01:31:36.799ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2240 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 302669b0-e7dd-49b8-8586-d2bc02e1fc04 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18112025-06-04T01:31:37.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
18122025-06-04T01:31:37.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18132025-06-04T01:31:37.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18142025-06-04T01:31:37.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
18152025-06-04T01:31:37.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18162025-06-04T01:31:37.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18172025-06-04T01:31:37.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18182025-06-04T01:31:37.735ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18192025-06-04T01:31:37.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18202025-06-04T01:31:37.737ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.452945ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18212025-06-04T01:31:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18222025-06-04T01:31:37.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18232025-06-04T01:31:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18242025-06-04T01:31:37.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18252025-06-04T01:31:37.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18262025-06-04T01:31:37.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18272025-06-04T01:31:37.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
18282025-06-04T01:31:37.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18292025-06-04T01:31:37.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18302025-06-04T01:31:37.755ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
18312025-06-04T01:31:37.755ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.404417ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
18322025-06-04T01:31:37.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c73f63b-6d46-4dc5-93c8-5f4cd188de31 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18332025-06-04T01:31:37.812ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c73f63b-6d46-4dc5-93c8-5f4cd188de31 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18342025-06-04T01:31:37.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5c73f63b-6d46-4dc5-93c8-5f4cd188de31 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18352025-06-04T01:31:38.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef72dfa7-ad6e-4b69-9316-22223f3a67d6 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18362025-06-04T01:31:38.828ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef72dfa7-ad6e-4b69-9316-22223f3a67d6 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18372025-06-04T01:31:38.830ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef72dfa7-ad6e-4b69-9316-22223f3a67d6 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18382025-06-04T01:31:39.843ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f70672f-78ed-48a0-b796-b36fd5af0ab4 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18392025-06-04T01:31:39.843ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f70672f-78ed-48a0-b796-b36fd5af0ab4 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18402025-06-04T01:31:39.845ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1903 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f70672f-78ed-48a0-b796-b36fd5af0ab4 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18412025-06-04T01:31:40.460ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 838 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 8b660fc4-168c-4536-83fa-4b3f73eaea2a response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
18422025-06-04T01:31:40.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b87f0472-e2df-4c5c-9b48-899e34907446 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18432025-06-04T01:31:40.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b87f0472-e2df-4c5c-9b48-899e34907446 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18442025-06-04T01:31:40.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2896 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b87f0472-e2df-4c5c-9b48-899e34907446 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18452025-06-04T01:31:41.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 762cf05c-1e34-4dd7-8f0d-d3466f60fb4c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18462025-06-04T01:31:41.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 762cf05c-1e34-4dd7-8f0d-d3466f60fb4c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18472025-06-04T01:31:41.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2531 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 762cf05c-1e34-4dd7-8f0d-d3466f60fb4c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18482025-06-04T01:31:42.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2c97c054-a6a0-4769-9ca9-1af9af30a24f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18492025-06-04T01:31:42.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2c97c054-a6a0-4769-9ca9-1af9af30a24f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18502025-06-04T01:31:42.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2c97c054-a6a0-4769-9ca9-1af9af30a24f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18512025-06-04T01:31:43.907ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4885f923-8c79-4b60-a6bc-2e5d0b66bb7a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18522025-06-04T01:31:43.907ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4885f923-8c79-4b60-a6bc-2e5d0b66bb7a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18532025-06-04T01:31:43.909ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4885f923-8c79-4b60-a6bc-2e5d0b66bb7a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18542025-06-04T01:31:44.923ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e73a4094-4376-49e6-a2d4-1d8e4971dcef roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18552025-06-04T01:31:44.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e73a4094-4376-49e6-a2d4-1d8e4971dcef resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18562025-06-04T01:31:44.925ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1879 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e73a4094-4376-49e6-a2d4-1d8e4971dcef response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18572025-06-04T01:31:45.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c015ff5-5522-4792-b0ee-9e2fd64baf1c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18582025-06-04T01:31:45.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c015ff5-5522-4792-b0ee-9e2fd64baf1c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18592025-06-04T01:31:45.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c015ff5-5522-4792-b0ee-9e2fd64baf1c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18602025-06-04T01:31:46.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eb9f6084-1942-4b47-9d6d-58e9b32f1e82 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18612025-06-04T01:31:46.956ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eb9f6084-1942-4b47-9d6d-58e9b32f1e82 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18622025-06-04T01:31:46.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2350 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = eb9f6084-1942-4b47-9d6d-58e9b32f1e82 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18632025-06-04T01:31:47.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
18642025-06-04T01:31:47.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18652025-06-04T01:31:47.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18662025-06-04T01:31:47.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
18672025-06-04T01:31:47.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18682025-06-04T01:31:47.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18692025-06-04T01:31:47.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18702025-06-04T01:31:47.734ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18712025-06-04T01:31:47.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18722025-06-04T01:31:47.736ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.865358ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18732025-06-04T01:31:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18742025-06-04T01:31:47.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18752025-06-04T01:31:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18762025-06-04T01:31:47.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18772025-06-04T01:31:47.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18782025-06-04T01:31:47.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18792025-06-04T01:31:47.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
18802025-06-04T01:31:47.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18812025-06-04T01:31:47.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18822025-06-04T01:31:47.752ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
18832025-06-04T01:31:47.752ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.339668ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
18842025-06-04T01:31:47.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 675eb85c-55dd-4d2c-b08c-cebe273036b5 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18852025-06-04T01:31:47.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 675eb85c-55dd-4d2c-b08c-cebe273036b5 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18862025-06-04T01:31:47.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2858 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 675eb85c-55dd-4d2c-b08c-cebe273036b5 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18872025-06-04T01:31:48.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd9449e2-8e82-427b-9750-fe05263a2aee roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18882025-06-04T01:31:48.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd9449e2-8e82-427b-9750-fe05263a2aee resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18892025-06-04T01:31:48.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1962 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd9449e2-8e82-427b-9750-fe05263a2aee response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18902025-06-04T01:31:50.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c132271-0e2e-4e6e-862e-311a5faf88df roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18912025-06-04T01:31:50.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c132271-0e2e-4e6e-862e-311a5faf88df resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18922025-06-04T01:31:50.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2468 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7c132271-0e2e-4e6e-862e-311a5faf88df response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18932025-06-04T01:31:50.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 599 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 83a52d0e-a089-40ac-85d7-9a43db562ccd response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
18942025-06-04T01:31:51.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06180c59-2985-4113-9522-0c7cb418aa7e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18952025-06-04T01:31:51.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06180c59-2985-4113-9522-0c7cb418aa7e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18962025-06-04T01:31:51.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2254 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06180c59-2985-4113-9522-0c7cb418aa7e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18972025-06-04T01:31:52.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cc7746af-659f-4f29-bcae-8b437b43abbd roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18982025-06-04T01:31:52.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cc7746af-659f-4f29-bcae-8b437b43abbd resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
18992025-06-04T01:31:52.036ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cc7746af-659f-4f29-bcae-8b437b43abbd response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19002025-06-04T01:31:53.049ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 745fa757-66fe-4014-9497-bc87171b005f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19012025-06-04T01:31:53.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 745fa757-66fe-4014-9497-bc87171b005f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19022025-06-04T01:31:53.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2670 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 745fa757-66fe-4014-9497-bc87171b005f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19032025-06-04T01:31:54.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4d0a6c97-11ac-45bf-8571-cf6371594c81 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19042025-06-04T01:31:54.066ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4d0a6c97-11ac-45bf-8571-cf6371594c81 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19052025-06-04T01:31:54.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2333 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4d0a6c97-11ac-45bf-8571-cf6371594c81 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19062025-06-04T01:31:55.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bae87e5b-ab5b-4d20-bf54-574832247978 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19072025-06-04T01:31:55.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bae87e5b-ab5b-4d20-bf54-574832247978 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19082025-06-04T01:31:55.083ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bae87e5b-ab5b-4d20-bf54-574832247978 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19092025-06-04T01:31:55.452ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:48583
19102025-06-04T01:31:55.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 0749b701-5ad1-4407-a65b-3a08c52df9dd roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
19112025-06-04T01:31:55.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 0749b701-5ad1-4407-a65b-3a08c52df9dd resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
19122025-06-04T01:31:55.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2870 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 0749b701-5ad1-4407-a65b-3a08c52df9dd response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
19132025-06-04T01:31:55.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = f0920d59-79c9-4a0d-8cc2-1159d710a77f roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
19142025-06-04T01:31:55.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = f0920d59-79c9-4a0d-8cc2-1159d710a77f resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
19152025-06-04T01:31:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1844 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = f0920d59-79c9-4a0d-8cc2-1159d710a77f response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
19162025-06-04T01:31:56.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f940c98e-3988-4158-8f1b-af72d16da4fe roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19172025-06-04T01:31:56.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f940c98e-3988-4158-8f1b-af72d16da4fe resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19182025-06-04T01:31:56.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f940c98e-3988-4158-8f1b-af72d16da4fe response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19192025-06-04T01:31:57.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e8ee682a-0ced-4956-ae7a-a610e2e90b4f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19202025-06-04T01:31:57.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e8ee682a-0ced-4956-ae7a-a610e2e90b4f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19212025-06-04T01:31:57.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2369 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e8ee682a-0ced-4956-ae7a-a610e2e90b4f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
19222025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
19232025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19242025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
19252025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
19262025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19272025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19282025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
19292025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
19302025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19312025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
19322025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
19332025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19342025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
19352025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19362025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19372025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19382025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
19392025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
19402025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19412025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19422025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
19432025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19442025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
19452025-06-04T01:31:57.732ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19462025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19472025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
19482025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
19492025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19502025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19512025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19522025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19532025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19542025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19552025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19562025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19572025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19582025-06-04T01:31:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19592025-06-04T01:31:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19602025-06-04T01:31:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19612025-06-04T01:31:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19622025-06-04T01:31:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19632025-06-04T01:31:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19642025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
19652025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
19662025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19672025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
19682025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
19692025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19702025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
19712025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19722025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
19732025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19742025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
19752025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19762025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19772025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19782025-06-04T01:31:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19792025-06-04T01:31:57.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19802025-06-04T01:31:57.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1854, tv_nsec: 830356801 } })
19812025-06-04T01:31:57.736ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19822025-06-04T01:31:57.736ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
19832025-06-04T01:31:57.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19842025-06-04T01:31:57.736ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19852025-06-04T01:31:57.737ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.752614ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19862025-06-04T01:31:57.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19872025-06-04T01:31:57.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19882025-06-04T01:31:57.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19892025-06-04T01:31:57.738ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19902025-06-04T01:31:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19912025-06-04T01:31:57.738ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19922025-06-04T01:31:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19932025-06-04T01:31:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19942025-06-04T01:31:57.739ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19952025-06-04T01:31:57.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19962025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19972025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19982025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19992025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20002025-06-04T01:31:57.742ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20012025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20022025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20032025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20042025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20052025-06-04T01:31:57.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20062025-06-04T01:31:57.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20072025-06-04T01:31:57.744ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1854, tv_nsec: 838643314 } })
20082025-06-04T01:31:57.744ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
20092025-06-04T01:31:57.745ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.236106ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
20102025-06-04T01:31:57.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20112025-06-04T01:31:57.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20122025-06-04T01:31:57.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20132025-06-04T01:31:57.748ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20142025-06-04T01:31:57.748ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 16.312738ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20152025-06-04T01:31:57.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20162025-06-04T01:31:57.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20172025-06-04T01:31:57.749ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 17.569714ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20182025-06-04T01:31:57.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20192025-06-04T01:31:57.750ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20202025-06-04T01:31:57.751ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20212025-06-04T01:31:57.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20222025-06-04T01:31:57.751ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20232025-06-04T01:31:57.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20242025-06-04T01:31:57.752ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20252025-06-04T01:31:57.752ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20262025-06-04T01:31:57.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20272025-06-04T01:31:57.752ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1854, tv_nsec: 846302319 } })
20282025-06-04T01:31:57.752ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
20292025-06-04T01:31:57.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20302025-06-04T01:31:57.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
20312025-06-04T01:31:57.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20322025-06-04T01:31:57.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20332025-06-04T01:31:57.768ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20342025-06-04T01:31:57.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20352025-06-04T01:31:57.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20362025-06-04T01:31:57.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20372025-06-04T01:31:57.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20382025-06-04T01:31:57.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20392025-06-04T01:31:57.776ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20402025-06-04T01:31:57.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20412025-06-04T01:31:57.778ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20422025-06-04T01:31:57.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20432025-06-04T01:31:57.781ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 47.176674ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
20442025-06-04T01:31:57.783ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20452025-06-04T01:31:57.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20462025-06-04T01:31:57.785ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 50.685162ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
20472025-06-04T01:31:57.785ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20482025-06-04T01:31:57.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20492025-06-04T01:31:57.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 51.776743ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20502025-06-04T01:31:57.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20512025-06-04T01:31:57.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20522025-06-04T01:31:57.793ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20532025-06-04T01:31:57.793ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20542025-06-04T01:31:57.793ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20552025-06-04T01:31:57.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20562025-06-04T01:31:57.796ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
20572025-06-04T01:31:57.796ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20582025-06-04T01:31:57.796ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2af8ca6d-97c6-4bd1-9ffa-abb3857a95dd", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
20592025-06-04T01:31:57.796ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20602025-06-04T01:31:57.797ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20612025-06-04T01:31:57.821ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
20622025-06-04T01:31:57.821ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20632025-06-04T01:31:57.821ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cfcb8f4-a385-4753-83d4-eadae178341f", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
20642025-06-04T01:31:57.821ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20652025-06-04T01:31:57.824ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20662025-06-04T01:31:57.848ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
20672025-06-04T01:31:57.848ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20682025-06-04T01:31:57.848ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "20de7da5-bdcc-44e7-9eaf-d30448dd2b13", "content-length": "26", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
20692025-06-04T01:31:57.848ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20702025-06-04T01:31:57.850ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
20712025-06-04T01:31:57.857ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20722025-06-04T01:31:57.857ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20732025-06-04T01:31:57.857ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dae6a39c-e488-43f2-8b66-b3d67cf9a6a6", "content-length": "372", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
20742025-06-04T01:31:57.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20752025-06-04T01:31:57.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20762025-06-04T01:31:57.859ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20772025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20782025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20792025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20802025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20812025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20822025-06-04T01:31:57.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20832025-06-04T01:31:57.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20842025-06-04T01:31:57.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20852025-06-04T01:31:57.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20862025-06-04T01:31:57.863ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20872025-06-04T01:31:57.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20882025-06-04T01:31:57.865ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20892025-06-04T01:31:57.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20902025-06-04T01:31:57.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20912025-06-04T01:31:57.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20922025-06-04T01:31:57.873ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
20932025-06-04T01:31:57.873ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 141.129852ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
20942025-06-04T01:31:57.888ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
20952025-06-04T01:31:57.888ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
20962025-06-04T01:31:57.888ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bb3e4e9a-9840-4e2c-a7f0-479bbc01f3cc", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
20972025-06-04T01:31:57.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20982025-06-04T01:31:57.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20992025-06-04T01:31:57.891ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21002025-06-04T01:31:57.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21012025-06-04T01:31:57.892ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 160.594888ms iteration = 28 status = {}
21022025-06-04T01:31:57.895ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21032025-06-04T01:31:57.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21042025-06-04T01:31:57.900ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
21052025-06-04T01:31:57.900ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21062025-06-04T01:31:57.900ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "940dad15-de8d-4496-82ec-fff81b2c77b1", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
21072025-06-04T01:31:57.902ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 170.056683ms iteration = 29 status = {}
21082025-06-04T01:31:57.906ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21092025-06-04T01:31:57.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21102025-06-04T01:31:57.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21112025-06-04T01:31:57.909ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21122025-06-04T01:31:57.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21132025-06-04T01:31:57.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
21142025-06-04T01:31:57.910ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21152025-06-04T01:31:57.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21162025-06-04T01:31:57.912ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 180.425455ms iteration = 28 status = 0
21172025-06-04T01:31:57.913ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21182025-06-04T01:31:57.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21192025-06-04T01:31:57.915ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21202025-06-04T01:31:57.915ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21212025-06-04T01:31:57.915ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 183.533062ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21222025-06-04T01:31:57.915ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 181.395012ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
21232025-06-04T01:31:57.916ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
21242025-06-04T01:31:57.916ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 184.078607ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
21252025-06-04T01:31:57.919ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
21262025-06-04T01:31:57.919ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 186.667909ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
21272025-06-04T01:31:57.922ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21282025-06-04T01:31:57.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21292025-06-04T01:31:57.925ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 192.864674ms iteration = 28 status = null
21302025-06-04T01:31:57.998ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 265.976446ms iteration = 31 status = {"physical_disks_added":0}
21312025-06-04T01:31:58.008ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21322025-06-04T01:31:58.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21332025-06-04T01:31:58.049ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21342025-06-04T01:31:58.049ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cb01e52-9c87-440a-8a3e-b96fc8c7aa2a", "content-length": "691", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
21352025-06-04T01:31:58.049ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
21362025-06-04T01:31:58.049ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
21372025-06-04T01:31:58.049ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
21382025-06-04T01:31:58.050ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 318.103559ms iteration = 29 status = {}
21392025-06-04T01:31:58.052ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
21402025-06-04T01:31:58.052ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21412025-06-04T01:31:58.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21422025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
21432025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
21442025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
21452025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
21462025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21472025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21482025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21492025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
21502025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
21512025-06-04T01:31:58.071ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
21522025-06-04T01:31:58.072ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
21532025-06-04T01:31:58.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21542025-06-04T01:31:58.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21552025-06-04T01:31:58.076ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
21562025-06-04T01:31:58.076ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
21572025-06-04T01:31:58.216ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "22904a43-0843-4d1b-b4c2-caa92d184ce8", "date": "Wed, 04 Jun 2025 01:31:57 GMT"} })
21582025-06-04T01:31:58.217ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
21592025-06-04T01:31:58.217ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
21602025-06-04T01:31:58.217ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c4fca85e-6348-4fbf-99a1-a86e3f92b395 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21612025-06-04T01:31:58.217ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c4fca85e-6348-4fbf-99a1-a86e3f92b395 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21622025-06-04T01:31:58.217ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "be64dd56-2024-4153-b20f-1f168e8d781b", "content-length": "682", "date": "Wed, 04 Jun 2025 01:31:58 GMT"} })
21632025-06-04T01:31:58.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21642025-06-04T01:31:58.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21652025-06-04T01:31:58.220ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3679 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c4fca85e-6348-4fbf-99a1-a86e3f92b395 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21662025-06-04T01:31:58.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21672025-06-04T01:31:58.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21682025-06-04T01:31:58.227ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21692025-06-04T01:31:58.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21702025-06-04T01:31:58.229ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21712025-06-04T01:31:58.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21722025-06-04T01:31:58.241ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 508.848823ms iteration = 28 status = {}
21732025-06-04T01:31:58.679ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
21742025-06-04T01:31:58.679ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
21752025-06-04T01:31:58.708ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
21762025-06-04T01:31:58.708ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:42932
21772025-06-04T01:31:58.708ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42932 req_id = eccc072f-6e77-49b1-a2f4-265a84bb545e roles = RoleSet { roles: {} } uri = /metrics/producers
21782025-06-04T01:31:58.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42932 req_id = eccc072f-6e77-49b1-a2f4-265a84bb545e resource = Database result = Ok(()) uri = /metrics/producers
21792025-06-04T01:31:58.746ZINFOnexus: assigned collector to new producer collector_id = b0f0765d-1e87-4904-b423-1a97eb2acce1 file = nexus/src/app/oximeter.rs:76 producer_id = 7716fd65-e841-4c70-8dfc-958c8d9211a5
21802025-06-04T01:31:58.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 38317 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:42932 req_id = eccc072f-6e77-49b1-a2f4-265a84bb545e response_code = 201 uri = /metrics/producers
21812025-06-04T01:31:58.747ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "eccc072f-6e77-49b1-a2f4-265a84bb545e", "content-length": "41", "date": "Wed, 04 Jun 2025 01:31:58 GMT"} })
21822025-06-04T01:31:58.748ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
21832025-06-04T01:31:59.239ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c17005-b90f-45f2-964a-080676509cb0 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21842025-06-04T01:31:59.239ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c17005-b90f-45f2-964a-080676509cb0 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21852025-06-04T01:31:59.242ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2734 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c17005-b90f-45f2-964a-080676509cb0 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21862025-06-04T01:32:00.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b3fbf4c0-404b-4d2b-a5c0-26c2da912cc2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21872025-06-04T01:32:00.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b3fbf4c0-404b-4d2b-a5c0-26c2da912cc2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21882025-06-04T01:32:00.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b3fbf4c0-404b-4d2b-a5c0-26c2da912cc2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21892025-06-04T01:32:00.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 548 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 89d7e1a3-2f50-42e7-a29c-ae47f2b429d7 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
21902025-06-04T01:32:01.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = fba3383a-a0fa-4c8f-be77-08adb7e66c67 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21912025-06-04T01:32:01.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = fba3383a-a0fa-4c8f-be77-08adb7e66c67 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21922025-06-04T01:32:01.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2586 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = fba3383a-a0fa-4c8f-be77-08adb7e66c67 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21932025-06-04T01:32:02.288ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 289e0a8c-aee0-486c-b379-1434fb375962 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21942025-06-04T01:32:02.288ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 289e0a8c-aee0-486c-b379-1434fb375962 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21952025-06-04T01:32:02.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 289e0a8c-aee0-486c-b379-1434fb375962 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21962025-06-04T01:32:03.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 34b4c58e-7dc7-434e-87b6-f75c655bcbfe roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21972025-06-04T01:32:03.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 34b4c58e-7dc7-434e-87b6-f75c655bcbfe resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21982025-06-04T01:32:03.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2483 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 34b4c58e-7dc7-434e-87b6-f75c655bcbfe response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
21992025-06-04T01:32:04.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 23a3e3c9-4659-4396-bb66-36dff44b2793 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22002025-06-04T01:32:04.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 23a3e3c9-4659-4396-bb66-36dff44b2793 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22012025-06-04T01:32:04.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2510 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 23a3e3c9-4659-4396-bb66-36dff44b2793 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22022025-06-04T01:32:05.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 62c4ceb2-24d9-4500-9959-b341d5499f34 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22032025-06-04T01:32:05.339ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 62c4ceb2-24d9-4500-9959-b341d5499f34 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22042025-06-04T01:32:05.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2391 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 62c4ceb2-24d9-4500-9959-b341d5499f34 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22052025-06-04T01:32:06.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bdb6b3ea-7369-442f-974e-dc0bf274a783 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22062025-06-04T01:32:06.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bdb6b3ea-7369-442f-974e-dc0bf274a783 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22072025-06-04T01:32:06.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bdb6b3ea-7369-442f-974e-dc0bf274a783 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22082025-06-04T01:32:07.370ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 28416c73-b3a3-43a3-a5ce-ab20d0d01e39 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22092025-06-04T01:32:07.370ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 28416c73-b3a3-43a3-a5ce-ab20d0d01e39 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22102025-06-04T01:32:07.372ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 28416c73-b3a3-43a3-a5ce-ab20d0d01e39 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22112025-06-04T01:32:07.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
22122025-06-04T01:32:07.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22132025-06-04T01:32:07.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22142025-06-04T01:32:07.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
22152025-06-04T01:32:07.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22162025-06-04T01:32:07.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22172025-06-04T01:32:07.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22182025-06-04T01:32:07.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22192025-06-04T01:32:07.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22202025-06-04T01:32:07.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22212025-06-04T01:32:07.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22222025-06-04T01:32:07.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22232025-06-04T01:32:07.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22242025-06-04T01:32:07.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
22252025-06-04T01:32:07.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22262025-06-04T01:32:07.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22272025-06-04T01:32:07.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
22282025-06-04T01:32:07.758ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.541705ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
22292025-06-04T01:32:07.772ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22302025-06-04T01:32:07.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22312025-06-04T01:32:07.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.393212ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22322025-06-04T01:32:08.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c85bb876-eb70-4e9d-9cca-29c57fc0cccf roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22332025-06-04T01:32:08.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c85bb876-eb70-4e9d-9cca-29c57fc0cccf resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22342025-06-04T01:32:08.388ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c85bb876-eb70-4e9d-9cca-29c57fc0cccf response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22352025-06-04T01:32:09.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b36a7607-5764-48b5-b2ce-0ec08d4a69e1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22362025-06-04T01:32:09.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b36a7607-5764-48b5-b2ce-0ec08d4a69e1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22372025-06-04T01:32:09.403ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2134 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b36a7607-5764-48b5-b2ce-0ec08d4a69e1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22382025-06-04T01:32:10.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac50532a-2396-4b79-9acb-46e6ebc14bac roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22392025-06-04T01:32:10.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac50532a-2396-4b79-9acb-46e6ebc14bac resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22402025-06-04T01:32:10.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ac50532a-2396-4b79-9acb-46e6ebc14bac response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22412025-06-04T01:32:10.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 600 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 62ebdc62-5c07-4065-9e07-e7de596b15a3 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
22422025-06-04T01:32:11.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 77bd5bbb-321b-46ba-a16e-c020dacb723c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22432025-06-04T01:32:11.433ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 77bd5bbb-321b-46ba-a16e-c020dacb723c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22442025-06-04T01:32:11.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 77bd5bbb-321b-46ba-a16e-c020dacb723c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22452025-06-04T01:32:12.447ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a4c5ed-06b0-4e1a-b7cc-330bfffbca1f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22462025-06-04T01:32:12.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a4c5ed-06b0-4e1a-b7cc-330bfffbca1f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22472025-06-04T01:32:12.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2539 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a4c5ed-06b0-4e1a-b7cc-330bfffbca1f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22482025-06-04T01:32:13.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 66c3608f-9348-45b2-bc0d-eb1807c21663 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22492025-06-04T01:32:13.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 66c3608f-9348-45b2-bc0d-eb1807c21663 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22502025-06-04T01:32:13.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 66c3608f-9348-45b2-bc0d-eb1807c21663 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22512025-06-04T01:32:14.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f70a9ee1-a21e-4ba2-adf6-e11e3e13f1c6 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22522025-06-04T01:32:14.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f70a9ee1-a21e-4ba2-adf6-e11e3e13f1c6 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22532025-06-04T01:32:14.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2033 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f70a9ee1-a21e-4ba2-adf6-e11e3e13f1c6 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22542025-06-04T01:32:15.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5ca65eec-e569-4cba-8dd2-d7d3f6555ae0 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22552025-06-04T01:32:15.496ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5ca65eec-e569-4cba-8dd2-d7d3f6555ae0 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22562025-06-04T01:32:15.499ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2825 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5ca65eec-e569-4cba-8dd2-d7d3f6555ae0 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22572025-06-04T01:32:16.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 80c9d9bd-63b2-4416-a69e-e055c0f3773e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22582025-06-04T01:32:16.513ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 80c9d9bd-63b2-4416-a69e-e055c0f3773e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22592025-06-04T01:32:16.515ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 80c9d9bd-63b2-4416-a69e-e055c0f3773e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22602025-06-04T01:32:17.529ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 50573cbd-61d4-47ee-89c9-2a03fbc4182e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22612025-06-04T01:32:17.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 50573cbd-61d4-47ee-89c9-2a03fbc4182e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22622025-06-04T01:32:17.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2570 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 50573cbd-61d4-47ee-89c9-2a03fbc4182e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22632025-06-04T01:32:17.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
22642025-06-04T01:32:17.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22652025-06-04T01:32:17.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22662025-06-04T01:32:17.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
22672025-06-04T01:32:17.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22682025-06-04T01:32:17.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22692025-06-04T01:32:17.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22702025-06-04T01:32:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22712025-06-04T01:32:17.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22722025-06-04T01:32:17.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22732025-06-04T01:32:17.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22742025-06-04T01:32:17.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22752025-06-04T01:32:17.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22762025-06-04T01:32:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
22772025-06-04T01:32:17.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22782025-06-04T01:32:17.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22792025-06-04T01:32:17.755ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
22802025-06-04T01:32:17.756ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.453936ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
22812025-06-04T01:32:17.765ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22822025-06-04T01:32:17.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22832025-06-04T01:32:17.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.679284ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22842025-06-04T01:32:18.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3ab2750d-18be-493c-b366-ca1a2020d7ee roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22852025-06-04T01:32:18.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3ab2750d-18be-493c-b366-ca1a2020d7ee resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22862025-06-04T01:32:18.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3ab2750d-18be-493c-b366-ca1a2020d7ee response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22872025-06-04T01:32:19.559ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c1213a15-e8d0-4d2a-8b5e-1821b4b0a4a9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22882025-06-04T01:32:19.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c1213a15-e8d0-4d2a-8b5e-1821b4b0a4a9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22892025-06-04T01:32:19.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2645 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c1213a15-e8d0-4d2a-8b5e-1821b4b0a4a9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22902025-06-04T01:32:20.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 738 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = b237f655-22e9-4f01-90bf-908a4f4b0119 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
22912025-06-04T01:32:20.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0c3e304-863f-430e-9e89-955562b46299 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22922025-06-04T01:32:20.575ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0c3e304-863f-430e-9e89-955562b46299 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22932025-06-04T01:32:20.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2536 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c0c3e304-863f-430e-9e89-955562b46299 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22942025-06-04T01:32:21.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c12ff7d7-528b-463a-880d-b4811dcff8fb roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22952025-06-04T01:32:21.591ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c12ff7d7-528b-463a-880d-b4811dcff8fb resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22962025-06-04T01:32:21.593ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2773 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c12ff7d7-528b-463a-880d-b4811dcff8fb response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22972025-06-04T01:32:22.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3b3676a5-fa30-43e1-8900-a199b101b89e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22982025-06-04T01:32:22.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3b3676a5-fa30-43e1-8900-a199b101b89e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
22992025-06-04T01:32:22.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2269 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3b3676a5-fa30-43e1-8900-a199b101b89e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23002025-06-04T01:32:23.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 89bdc9da-eb6c-4540-9aa6-a4677c5363c9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23012025-06-04T01:32:23.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 89bdc9da-eb6c-4540-9aa6-a4677c5363c9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23022025-06-04T01:32:23.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 89bdc9da-eb6c-4540-9aa6-a4677c5363c9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23032025-06-04T01:32:24.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cf857951-6a6c-4e0a-aa7e-dba78075fc4d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23042025-06-04T01:32:24.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cf857951-6a6c-4e0a-aa7e-dba78075fc4d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23052025-06-04T01:32:24.640ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2689 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cf857951-6a6c-4e0a-aa7e-dba78075fc4d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23062025-06-04T01:32:25.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a8c55b7-df8c-4fd7-b4e7-0d9ad6417c4b roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
23072025-06-04T01:32:25.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a8c55b7-df8c-4fd7-b4e7-0d9ad6417c4b resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
23082025-06-04T01:32:25.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3236 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a8c55b7-df8c-4fd7-b4e7-0d9ad6417c4b response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
23092025-06-04T01:32:25.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 57a8b14c-7d58-4778-9cc1-5787c857a4c7 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
23102025-06-04T01:32:25.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 57a8b14c-7d58-4778-9cc1-5787c857a4c7 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
23112025-06-04T01:32:25.458ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2529 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 57a8b14c-7d58-4778-9cc1-5787c857a4c7 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
23122025-06-04T01:32:25.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97bbad64-5ca6-4ea5-9b62-affde7a8261d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23132025-06-04T01:32:25.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97bbad64-5ca6-4ea5-9b62-affde7a8261d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23142025-06-04T01:32:25.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 97bbad64-5ca6-4ea5-9b62-affde7a8261d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23152025-06-04T01:32:26.669ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef4590d0-b1d6-491f-abd8-92135101027c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23162025-06-04T01:32:26.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef4590d0-b1d6-491f-abd8-92135101027c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23172025-06-04T01:32:26.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2158 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ef4590d0-b1d6-491f-abd8-92135101027c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23182025-06-04T01:32:27.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1a462159-33de-4085-b184-1a527740b318 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23192025-06-04T01:32:27.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1a462159-33de-4085-b184-1a527740b318 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23202025-06-04T01:32:27.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2842 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1a462159-33de-4085-b184-1a527740b318 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
23212025-06-04T01:32:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
23222025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
23232025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23242025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
23252025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
23262025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23272025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
23282025-06-04T01:32:27.732ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:22:27.732049195 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
23292025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
23302025-06-04T01:32:27.732ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
23312025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
23322025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.26\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
23332025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
23342025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
23352025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23362025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
23372025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
23382025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
23392025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
23402025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23412025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
23422025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
23432025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
23442025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23452025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23462025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
23472025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23482025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Timeout
23492025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
23502025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23512025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23522025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
23532025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
23542025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
23552025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23562025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
23572025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23582025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
23592025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
23602025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23612025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
23622025-06-04T01:32:27.732ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
23632025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23642025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
23652025-06-04T01:32:27.732ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
23662025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
23672025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
23682025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23692025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
23702025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
23712025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23722025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23732025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23742025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23752025-06-04T01:32:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23762025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23772025-06-04T01:32:27.733ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1884, tv_nsec: 827125141 } })
23782025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23792025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
23802025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
23812025-06-04T01:32:27.733ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1884, tv_nsec: 827175132 } })
23822025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23832025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23842025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23852025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23862025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23872025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23882025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23892025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23902025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
23912025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23922025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23932025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23942025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23952025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
23962025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
23972025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
23982025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.495042ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
23992025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
24002025-06-04T01:32:27.733ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.741399ms iteration = 15 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
24012025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
24022025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
24032025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24042025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
24052025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
24062025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
24072025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 15 reason = Timeout
24082025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
24092025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 29 reason = Timeout
24102025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
24112025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
24122025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
24132025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
24142025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24152025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 15 reason = Timeout
24162025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
24172025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24182025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
24192025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24202025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
24212025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
24222025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
24232025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
24242025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24252025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
24262025-06-04T01:32:27.734ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24272025-06-04T01:32:27.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24282025-06-04T01:32:27.735ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24292025-06-04T01:32:27.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24302025-06-04T01:32:27.736ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1884, tv_nsec: 830626659 } })
24312025-06-04T01:32:27.736ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.351062ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
24322025-06-04T01:32:27.736ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
24332025-06-04T01:32:27.736ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:22:27.732049195 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
24342025-06-04T01:32:27.736ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.721183ms iteration = 15 status = {"expiration":"2025-06-04T01:22:27.732049195Z","pruned":[]}
24352025-06-04T01:32:27.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24362025-06-04T01:32:27.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24372025-06-04T01:32:27.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24382025-06-04T01:32:27.738ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24392025-06-04T01:32:27.738ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24402025-06-04T01:32:27.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24412025-06-04T01:32:27.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24422025-06-04T01:32:27.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
24432025-06-04T01:32:27.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24442025-06-04T01:32:27.740ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1884, tv_nsec: 834737584 } })
24452025-06-04T01:32:27.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24462025-06-04T01:32:27.740ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
24472025-06-04T01:32:27.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24482025-06-04T01:32:27.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24492025-06-04T01:32:27.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24502025-06-04T01:32:27.741ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24512025-06-04T01:32:27.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24522025-06-04T01:32:27.743ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
24532025-06-04T01:32:27.743ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24542025-06-04T01:32:27.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24552025-06-04T01:32:27.743ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24562025-06-04T01:32:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24572025-06-04T01:32:27.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24582025-06-04T01:32:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24592025-06-04T01:32:27.744ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24602025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24612025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24622025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24632025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24642025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24652025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24662025-06-04T01:32:27.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24672025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
24682025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
24692025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 14.131587ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
24702025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
24712025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24722025-06-04T01:32:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24732025-06-04T01:32:27.751ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 16.836872ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
24742025-06-04T01:32:27.752ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
24752025-06-04T01:32:27.752ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 18.107979ms iteration = 15 status = {"by_rx":{},"error":null}
24762025-06-04T01:32:27.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24772025-06-04T01:32:27.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24782025-06-04T01:32:27.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 23.703275ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24792025-06-04T01:32:27.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24802025-06-04T01:32:27.758ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24812025-06-04T01:32:27.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24822025-06-04T01:32:27.759ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1884, tv_nsec: 853674016 } })
24832025-06-04T01:32:27.759ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24842025-06-04T01:32:27.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24852025-06-04T01:32:27.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24862025-06-04T01:32:27.759ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
24872025-06-04T01:32:27.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24882025-06-04T01:32:27.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24892025-06-04T01:32:27.764ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
24902025-06-04T01:32:27.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
24912025-06-04T01:32:27.766ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24922025-06-04T01:32:27.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24932025-06-04T01:32:27.771ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24942025-06-04T01:32:27.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24952025-06-04T01:32:27.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24962025-06-04T01:32:27.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24972025-06-04T01:32:27.774ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24982025-06-04T01:32:27.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24992025-06-04T01:32:27.782ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25002025-06-04T01:32:27.782ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
25012025-06-04T01:32:27.782ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25022025-06-04T01:32:27.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25032025-06-04T01:32:27.785ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
25042025-06-04T01:32:27.786ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
25052025-06-04T01:32:27.786ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 52.991058ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
25062025-06-04T01:32:27.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.949758ms iteration = 29 status = {"errors":[],"finish_invoked_ok":[]}
25072025-06-04T01:32:27.787ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 53.037738ms iteration = 15 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
25082025-06-04T01:32:27.791ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25092025-06-04T01:32:27.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25102025-06-04T01:32:27.791ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25112025-06-04T01:32:27.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25122025-06-04T01:32:27.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25132025-06-04T01:32:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25142025-06-04T01:32:27.801ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25152025-06-04T01:32:27.802ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25162025-06-04T01:32:27.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25172025-06-04T01:32:27.803ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 69.658485ms iteration = 29 status = {"errors":[],"requests_created_ok":[]}
25182025-06-04T01:32:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
25192025-06-04T01:32:27.804ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25202025-06-04T01:32:27.804ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25212025-06-04T01:32:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25222025-06-04T01:32:27.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25232025-06-04T01:32:27.806ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25242025-06-04T01:32:27.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25252025-06-04T01:32:27.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25262025-06-04T01:32:27.807ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25272025-06-04T01:32:27.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25282025-06-04T01:32:27.808ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25292025-06-04T01:32:27.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25302025-06-04T01:32:27.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25312025-06-04T01:32:27.810ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25322025-06-04T01:32:27.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25332025-06-04T01:32:27.812ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
25342025-06-04T01:32:27.812ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 80.058367ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
25352025-06-04T01:32:27.812ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
25362025-06-04T01:32:27.812ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
25372025-06-04T01:32:27.812ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.15317ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
25382025-06-04T01:32:27.817ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
25392025-06-04T01:32:27.817ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.542511ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
25402025-06-04T01:32:27.818ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25412025-06-04T01:32:27.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25422025-06-04T01:32:27.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25432025-06-04T01:32:27.823ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
25442025-06-04T01:32:27.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25452025-06-04T01:32:27.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25462025-06-04T01:32:27.838ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25472025-06-04T01:32:27.838ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
25482025-06-04T01:32:27.838ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.632903ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
25492025-06-04T01:32:27.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25502025-06-04T01:32:27.839ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25512025-06-04T01:32:27.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25522025-06-04T01:32:27.839ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.345267ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
25532025-06-04T01:32:27.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25542025-06-04T01:32:27.840ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25552025-06-04T01:32:27.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25562025-06-04T01:32:27.842ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
25572025-06-04T01:32:27.842ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25582025-06-04T01:32:27.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25592025-06-04T01:32:27.842ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25602025-06-04T01:32:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
25612025-06-04T01:32:27.845ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25622025-06-04T01:32:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25632025-06-04T01:32:27.846ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25642025-06-04T01:32:27.846ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
25652025-06-04T01:32:27.846ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 114.318039ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
25662025-06-04T01:32:27.847ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.166863ms iteration = 29 status = 0
25672025-06-04T01:32:27.848ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
25682025-06-04T01:32:27.848ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
25692025-06-04T01:32:27.848ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
25702025-06-04T01:32:27.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25712025-06-04T01:32:27.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25722025-06-04T01:32:27.849ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 117.693404ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
25732025-06-04T01:32:27.852ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
25742025-06-04T01:32:27.852ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
25752025-06-04T01:32:27.852ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
25762025-06-04T01:32:27.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25772025-06-04T01:32:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25782025-06-04T01:32:27.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25792025-06-04T01:32:27.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25802025-06-04T01:32:27.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25812025-06-04T01:32:27.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25822025-06-04T01:32:27.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
25832025-06-04T01:32:27.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
25842025-06-04T01:32:27.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 133.625931ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
25852025-06-04T01:32:27.883ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
25862025-06-04T01:32:27.883ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 151.571264ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
25872025-06-04T01:32:27.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25882025-06-04T01:32:27.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25892025-06-04T01:32:27.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25902025-06-04T01:32:27.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25912025-06-04T01:32:27.894ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25922025-06-04T01:32:27.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25932025-06-04T01:32:27.902ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
25942025-06-04T01:32:27.902ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
25952025-06-04T01:32:27.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25962025-06-04T01:32:27.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25972025-06-04T01:32:27.918ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
25982025-06-04T01:32:27.918ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25992025-06-04T01:32:27.918ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1cef083-a832-4a84-b3f2-c8829c97e81d", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:27 GMT"} })
26002025-06-04T01:32:27.918ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26012025-06-04T01:32:27.920ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 188.346417ms iteration = 32 status = {"physical_disks_added":0}
26022025-06-04T01:32:27.921ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26032025-06-04T01:32:27.965ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
26042025-06-04T01:32:27.965ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 233.309339ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
26052025-06-04T01:32:28.005ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
26062025-06-04T01:32:28.007ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
26072025-06-04T01:32:28.051ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
26082025-06-04T01:32:28.051ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26092025-06-04T01:32:28.051ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bccbfa2-c31c-49af-91bf-2e807325cf8f", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:27 GMT"} })
26102025-06-04T01:32:28.051ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26112025-06-04T01:32:28.053ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26122025-06-04T01:32:28.053ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
26132025-06-04T01:32:28.053ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26142025-06-04T01:32:28.054ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5de57675-4427-4c69-9d9a-d75f6ad6d90c", "content-length": "372", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
26152025-06-04T01:32:28.054ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26162025-06-04T01:32:28.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26172025-06-04T01:32:28.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26182025-06-04T01:32:28.089ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
26192025-06-04T01:32:28.089ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
26202025-06-04T01:32:28.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26212025-06-04T01:32:28.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26222025-06-04T01:32:28.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26232025-06-04T01:32:28.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26242025-06-04T01:32:28.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26252025-06-04T01:32:28.094ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
26262025-06-04T01:32:28.094ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
26272025-06-04T01:32:28.094ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
26282025-06-04T01:32:28.094ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 362.633509ms iteration = 15 status = {"generation":1}
26292025-06-04T01:32:28.098ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
26302025-06-04T01:32:28.098ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
26312025-06-04T01:32:28.098ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ab838b8e-c250-453e-828d-f6d435dba8ae", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
26322025-06-04T01:32:28.098ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26332025-06-04T01:32:28.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26342025-06-04T01:32:28.101ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26352025-06-04T01:32:28.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26362025-06-04T01:32:28.106ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26372025-06-04T01:32:28.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26382025-06-04T01:32:28.112ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
26392025-06-04T01:32:28.112ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26402025-06-04T01:32:28.113ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1db5b344-95ca-4d74-b291-23aea2a61ec4", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
26412025-06-04T01:32:28.114ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 382.196209ms iteration = 30 status = {}
26422025-06-04T01:32:28.118ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
26432025-06-04T01:32:28.118ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26442025-06-04T01:32:28.118ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "100128bc-f01c-4ef2-b1b0-34f8edd2cfca", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:27 GMT"} })
26452025-06-04T01:32:28.118ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26462025-06-04T01:32:28.120ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
26472025-06-04T01:32:28.132ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26482025-06-04T01:32:28.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26492025-06-04T01:32:28.135ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26502025-06-04T01:32:28.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26512025-06-04T01:32:28.137ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
26522025-06-04T01:32:28.137ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
26532025-06-04T01:32:28.137ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
26542025-06-04T01:32:28.137ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 405.517163ms iteration = 15 status = {"generation":2}
26552025-06-04T01:32:28.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26562025-06-04T01:32:28.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26572025-06-04T01:32:28.170ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26582025-06-04T01:32:28.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26592025-06-04T01:32:28.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26602025-06-04T01:32:28.200ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26612025-06-04T01:32:28.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26622025-06-04T01:32:28.209ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26632025-06-04T01:32:28.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26642025-06-04T01:32:28.211ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 479.092659ms iteration = 29 status = null
26652025-06-04T01:32:28.233ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26662025-06-04T01:32:28.235ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26672025-06-04T01:32:28.235ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26682025-06-04T01:32:28.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26692025-06-04T01:32:28.259ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 527.356143ms iteration = 29 status = {}
26702025-06-04T01:32:28.274ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
26712025-06-04T01:32:28.274ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2009a49a-3cf2-4aa7-9c25-90f40c2d1656", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26722025-06-04T01:32:28.274ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26732025-06-04T01:32:28.276ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "da435c32-c38c-4172-9195-61c4baa5ed51", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26742025-06-04T01:32:28.276ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26752025-06-04T01:32:28.278ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26762025-06-04T01:32:28.278ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 546.374457ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
26772025-06-04T01:32:28.285ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
26782025-06-04T01:32:28.285ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "509f2dcd-f284-4242-8537-f6f05b6e313c", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26792025-06-04T01:32:28.285ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26802025-06-04T01:32:28.287ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "bae3dd7f-577b-4194-9247-4fe638cf9acf", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26812025-06-04T01:32:28.287ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26822025-06-04T01:32:28.289ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "25c7ee21-a144-442d-a919-a165c1e35191", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26832025-06-04T01:32:28.289ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26842025-06-04T01:32:28.291ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26852025-06-04T01:32:28.291ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 559.760852ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
26862025-06-04T01:32:28.357ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26872025-06-04T01:32:28.358ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f62322f5-b9fb-473d-891e-863d552ad2ec", "content-length": "691", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
26882025-06-04T01:32:28.358ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
26892025-06-04T01:32:28.358ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26902025-06-04T01:32:28.358ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26912025-06-04T01:32:28.358ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 626.29091ms iteration = 30 status = {}
26922025-06-04T01:32:28.363ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26932025-06-04T01:32:28.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26942025-06-04T01:32:28.365ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
26952025-06-04T01:32:28.365ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26962025-06-04T01:32:28.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26972025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26982025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
26992025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
27002025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27012025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27022025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27032025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27042025-06-04T01:32:28.377ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27052025-06-04T01:32:28.378ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27062025-06-04T01:32:28.378ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27072025-06-04T01:32:28.378ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
27082025-06-04T01:32:28.378ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27092025-06-04T01:32:28.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27102025-06-04T01:32:28.380ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
27112025-06-04T01:32:28.380ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27122025-06-04T01:32:28.470ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "dd6601e6-633b-4303-b6f7-f65fae8b564c", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
27132025-06-04T01:32:28.470ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
27142025-06-04T01:32:28.470ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27152025-06-04T01:32:28.470ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9a24102-e048-4990-8f27-b5478a656690", "content-length": "682", "date": "Wed, 04 Jun 2025 01:32:28 GMT"} })
27162025-06-04T01:32:28.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27172025-06-04T01:32:28.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27182025-06-04T01:32:28.474ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27192025-06-04T01:32:28.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27202025-06-04T01:32:28.477ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27212025-06-04T01:32:28.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27222025-06-04T01:32:28.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27232025-06-04T01:32:28.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27242025-06-04T01:32:28.486ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 754.324934ms iteration = 29 status = {}
27252025-06-04T01:32:28.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 03df3e8b-7a22-4faa-8dde-2d34c8a60537 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27262025-06-04T01:32:28.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 03df3e8b-7a22-4faa-8dde-2d34c8a60537 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27272025-06-04T01:32:28.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2665 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 03df3e8b-7a22-4faa-8dde-2d34c8a60537 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27282025-06-04T01:32:29.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 978a5407-85bd-48e3-9e60-38dbfc3a51a3 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27292025-06-04T01:32:29.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 978a5407-85bd-48e3-9e60-38dbfc3a51a3 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27302025-06-04T01:32:29.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 4627 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 978a5407-85bd-48e3-9e60-38dbfc3a51a3 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27312025-06-04T01:32:30.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 702 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = e43739cb-0738-43b7-804b-6b892766a089 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
27322025-06-04T01:32:30.755ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0d4ebd86-f7a1-457f-9339-6beec9a3cd72 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27332025-06-04T01:32:30.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0d4ebd86-f7a1-457f-9339-6beec9a3cd72 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27342025-06-04T01:32:30.757ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0d4ebd86-f7a1-457f-9339-6beec9a3cd72 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27352025-06-04T01:32:31.770ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8635c4f3-5281-4e94-a3a9-53da30b080a9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27362025-06-04T01:32:31.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8635c4f3-5281-4e94-a3a9-53da30b080a9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27372025-06-04T01:32:31.773ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3042 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8635c4f3-5281-4e94-a3a9-53da30b080a9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27382025-06-04T01:32:32.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ccb6b3ef-6b08-4511-8ec3-5f36b4f41a5d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27392025-06-04T01:32:32.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ccb6b3ef-6b08-4511-8ec3-5f36b4f41a5d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27402025-06-04T01:32:32.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2617 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ccb6b3ef-6b08-4511-8ec3-5f36b4f41a5d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27412025-06-04T01:32:33.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ed6ec4d5-5ba1-4480-b2b8-398e15a1db98 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27422025-06-04T01:32:33.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ed6ec4d5-5ba1-4480-b2b8-398e15a1db98 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27432025-06-04T01:32:33.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2923 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ed6ec4d5-5ba1-4480-b2b8-398e15a1db98 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27442025-06-04T01:32:34.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 991efb75-df26-4459-95ee-f23ecea71893 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27452025-06-04T01:32:34.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 991efb75-df26-4459-95ee-f23ecea71893 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27462025-06-04T01:32:34.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2652 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 991efb75-df26-4459-95ee-f23ecea71893 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27472025-06-04T01:32:35.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9d7ecc18-ad96-46d3-aa66-2016a7a7ab92 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27482025-06-04T01:32:35.834ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9d7ecc18-ad96-46d3-aa66-2016a7a7ab92 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27492025-06-04T01:32:35.836ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2415 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 9d7ecc18-ad96-46d3-aa66-2016a7a7ab92 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27502025-06-04T01:32:36.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ea7f827d-6ef1-4782-974b-8a638c609930 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27512025-06-04T01:32:36.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ea7f827d-6ef1-4782-974b-8a638c609930 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27522025-06-04T01:32:36.851ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2416 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ea7f827d-6ef1-4782-974b-8a638c609930 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27532025-06-04T01:32:37.730ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
27542025-06-04T01:32:37.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27552025-06-04T01:32:37.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27562025-06-04T01:32:37.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
27572025-06-04T01:32:37.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27582025-06-04T01:32:37.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27592025-06-04T01:32:37.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27602025-06-04T01:32:37.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27612025-06-04T01:32:37.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27622025-06-04T01:32:37.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27632025-06-04T01:32:37.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27642025-06-04T01:32:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27652025-06-04T01:32:37.765ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27662025-06-04T01:32:37.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27672025-06-04T01:32:37.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27682025-06-04T01:32:37.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
27692025-06-04T01:32:37.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27702025-06-04T01:32:37.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27712025-06-04T01:32:37.785ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
27722025-06-04T01:32:37.785ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.687285ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
27732025-06-04T01:32:37.798ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.818606ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27742025-06-04T01:32:37.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99cfe933-90d3-4ec3-a8c1-fe21bfc98d32 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27752025-06-04T01:32:37.866ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99cfe933-90d3-4ec3-a8c1-fe21bfc98d32 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27762025-06-04T01:32:37.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1850 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 99cfe933-90d3-4ec3-a8c1-fe21bfc98d32 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27772025-06-04T01:32:38.881ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 11d9f676-2639-4ebd-9d15-4854b41cac35 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27782025-06-04T01:32:38.881ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 11d9f676-2639-4ebd-9d15-4854b41cac35 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27792025-06-04T01:32:38.883ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2205 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 11d9f676-2639-4ebd-9d15-4854b41cac35 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27802025-06-04T01:32:39.897ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c256c178-290c-44d6-8659-a885dee929a4 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27812025-06-04T01:32:39.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c256c178-290c-44d6-8659-a885dee929a4 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27822025-06-04T01:32:39.899ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2288 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c256c178-290c-44d6-8659-a885dee929a4 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27832025-06-04T01:32:40.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 5d7460f9-f277-4098-a72e-d6a27dbbff28 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
27842025-06-04T01:32:40.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 5d7460f9-f277-4098-a72e-d6a27dbbff28 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
27852025-06-04T01:32:40.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3280 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 5d7460f9-f277-4098-a72e-d6a27dbbff28 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
27862025-06-04T01:32:40.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 43c1de5c-9de8-463a-b853-8f0330d99aed roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
27872025-06-04T01:32:40.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 43c1de5c-9de8-463a-b853-8f0330d99aed resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
27882025-06-04T01:32:40.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2550 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 43c1de5c-9de8-463a-b853-8f0330d99aed response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
27892025-06-04T01:32:40.458ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 640 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 3e360197-ddca-4ac7-aa17-190aa25fb645 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
27902025-06-04T01:32:40.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2501714b-24b3-4279-8a2e-058726bccddc roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27912025-06-04T01:32:40.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2501714b-24b3-4279-8a2e-058726bccddc resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27922025-06-04T01:32:40.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2501714b-24b3-4279-8a2e-058726bccddc response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27932025-06-04T01:32:41.930ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06dcb723-5d92-42a6-bcdf-f48b9a70a14c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27942025-06-04T01:32:41.930ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06dcb723-5d92-42a6-bcdf-f48b9a70a14c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27952025-06-04T01:32:41.932ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2463 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 06dcb723-5d92-42a6-bcdf-f48b9a70a14c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27962025-06-04T01:32:42.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0e0f432b-913b-4f7a-83fd-45dbbea656bc roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27972025-06-04T01:32:42.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0e0f432b-913b-4f7a-83fd-45dbbea656bc resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27982025-06-04T01:32:42.948ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2596 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0e0f432b-913b-4f7a-83fd-45dbbea656bc response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
27992025-06-04T01:32:43.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0537110b-e2ab-4e54-b155-20ed7a1d82a1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28002025-06-04T01:32:43.961ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0537110b-e2ab-4e54-b155-20ed7a1d82a1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28012025-06-04T01:32:43.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2437 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0537110b-e2ab-4e54-b155-20ed7a1d82a1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28022025-06-04T01:32:44.976ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95d4ab03-32e8-4811-8835-f83c8b69cb16 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28032025-06-04T01:32:44.976ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95d4ab03-32e8-4811-8835-f83c8b69cb16 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28042025-06-04T01:32:44.978ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2235 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 95d4ab03-32e8-4811-8835-f83c8b69cb16 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28052025-06-04T01:32:45.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 05e61a1f-0d16-4e8b-b565-1b01509f5103 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28062025-06-04T01:32:45.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 05e61a1f-0d16-4e8b-b565-1b01509f5103 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28072025-06-04T01:32:45.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2476 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 05e61a1f-0d16-4e8b-b565-1b01509f5103 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28082025-06-04T01:32:47.009ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d74fe633-55a8-4404-a5e7-8c090251298a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28092025-06-04T01:32:47.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d74fe633-55a8-4404-a5e7-8c090251298a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28102025-06-04T01:32:47.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2575 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d74fe633-55a8-4404-a5e7-8c090251298a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28112025-06-04T01:32:47.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
28122025-06-04T01:32:47.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28132025-06-04T01:32:47.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28142025-06-04T01:32:47.731ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
28152025-06-04T01:32:47.731ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28162025-06-04T01:32:47.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28172025-06-04T01:32:47.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28182025-06-04T01:32:47.734ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28192025-06-04T01:32:47.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28202025-06-04T01:32:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28212025-06-04T01:32:47.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28222025-06-04T01:32:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28232025-06-04T01:32:47.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28242025-06-04T01:32:47.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28252025-06-04T01:32:47.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28262025-06-04T01:32:47.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
28272025-06-04T01:32:47.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28282025-06-04T01:32:47.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28292025-06-04T01:32:47.755ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
28302025-06-04T01:32:47.755ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.476796ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
28312025-06-04T01:32:47.767ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.513037ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28322025-06-04T01:32:48.026ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1b967ea3-30c2-4042-a8eb-77e5c20e87aa roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28332025-06-04T01:32:48.026ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1b967ea3-30c2-4042-a8eb-77e5c20e87aa resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28342025-06-04T01:32:48.028ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2488 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1b967ea3-30c2-4042-a8eb-77e5c20e87aa response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28352025-06-04T01:32:49.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = baa4de15-27cb-4d45-858e-4b9b40f1983c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28362025-06-04T01:32:49.042ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = baa4de15-27cb-4d45-858e-4b9b40f1983c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28372025-06-04T01:32:49.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2379 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = baa4de15-27cb-4d45-858e-4b9b40f1983c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28382025-06-04T01:32:50.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0adba6db-5fa6-4aee-af26-ed13dafa4c9f roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28392025-06-04T01:32:50.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0adba6db-5fa6-4aee-af26-ed13dafa4c9f resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28402025-06-04T01:32:50.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2092 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 0adba6db-5fa6-4aee-af26-ed13dafa4c9f response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28412025-06-04T01:32:50.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 890 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 36bc65f0-5c93-4c81-9ca3-6f2303d1f13a response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
28422025-06-04T01:32:51.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 18ccb9a3-d277-4fae-92b2-a71c98b3dd9d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28432025-06-04T01:32:51.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 18ccb9a3-d277-4fae-92b2-a71c98b3dd9d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28442025-06-04T01:32:51.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2738 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 18ccb9a3-d277-4fae-92b2-a71c98b3dd9d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28452025-06-04T01:32:52.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4dabaa48-62e3-4e7c-95c0-22c1055c87d5 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28462025-06-04T01:32:52.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4dabaa48-62e3-4e7c-95c0-22c1055c87d5 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28472025-06-04T01:32:52.091ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2466 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4dabaa48-62e3-4e7c-95c0-22c1055c87d5 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28482025-06-04T01:32:53.108ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f8690cfd-cac2-4a8c-ab8b-cfc561439d49 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28492025-06-04T01:32:53.108ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f8690cfd-cac2-4a8c-ab8b-cfc561439d49 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28502025-06-04T01:32:53.110ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f8690cfd-cac2-4a8c-ab8b-cfc561439d49 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28512025-06-04T01:32:54.123ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 09f55434-819b-48cb-90a5-3fa8641b5d98 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28522025-06-04T01:32:54.123ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 09f55434-819b-48cb-90a5-3fa8641b5d98 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28532025-06-04T01:32:54.125ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2427 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 09f55434-819b-48cb-90a5-3fa8641b5d98 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28542025-06-04T01:32:55.139ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a34739f3-121f-4112-8300-1fdd16eefbb6 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28552025-06-04T01:32:55.139ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a34739f3-121f-4112-8300-1fdd16eefbb6 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28562025-06-04T01:32:55.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a34739f3-121f-4112-8300-1fdd16eefbb6 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28572025-06-04T01:32:55.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a329991-1207-4882-92bc-379522f772fb roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28582025-06-04T01:32:55.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a329991-1207-4882-92bc-379522f772fb resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28592025-06-04T01:32:55.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3778 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = 3a329991-1207-4882-92bc-379522f772fb response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
28602025-06-04T01:32:55.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = ab01efa1-3e71-4db2-8dc1-0f6242c8a424 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28612025-06-04T01:32:55.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = ab01efa1-3e71-4db2-8dc1-0f6242c8a424 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28622025-06-04T01:32:55.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2059 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48583 req_id = ab01efa1-3e71-4db2-8dc1-0f6242c8a424 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
28632025-06-04T01:32:56.154ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70fb6af8-5baf-4059-bbe3-fa7612bb865b roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28642025-06-04T01:32:56.154ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70fb6af8-5baf-4059-bbe3-fa7612bb865b resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28652025-06-04T01:32:56.156ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2744 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 70fb6af8-5baf-4059-bbe3-fa7612bb865b response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28662025-06-04T01:32:57.171ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8d500c73-1bae-4e72-9a4f-eaac51f2e2c2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28672025-06-04T01:32:57.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8d500c73-1bae-4e72-9a4f-eaac51f2e2c2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28682025-06-04T01:32:57.173ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1964 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8d500c73-1bae-4e72-9a4f-eaac51f2e2c2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
28692025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
28702025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
28712025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
28722025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28732025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28742025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28752025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
28762025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28772025-06-04T01:32:57.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28782025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28792025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
28802025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28812025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
28822025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
28832025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28842025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
28852025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28862025-06-04T01:32:57.732ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28872025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28882025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
28892025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28902025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
28912025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
28922025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
28932025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28942025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28952025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
28962025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28972025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
28982025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28992025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
29002025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29012025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29022025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29032025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29042025-06-04T01:32:57.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29052025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29062025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29072025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29082025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29092025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29102025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29112025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
29122025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
29132025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
29142025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29152025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 30 reason = Timeout
29162025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
29172025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
29182025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29192025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
29202025-06-04T01:32:57.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
29212025-06-04T01:32:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
29222025-06-04T01:32:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
29232025-06-04T01:32:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29242025-06-04T01:32:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
29252025-06-04T01:32:57.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29262025-06-04T01:32:57.734ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1914, tv_nsec: 829470767 } })
29272025-06-04T01:32:57.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29282025-06-04T01:32:57.735ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
29292025-06-04T01:32:57.735ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1914, tv_nsec: 829970761 } })
29302025-06-04T01:32:57.735ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29312025-06-04T01:32:57.735ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
29322025-06-04T01:32:57.736ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.520967ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
29332025-06-04T01:32:57.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29342025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
29352025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29362025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29372025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29382025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29392025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29402025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29412025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29422025-06-04T01:32:57.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29432025-06-04T01:32:57.741ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442025-06-04T01:32:57.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29452025-06-04T01:32:57.743ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29462025-06-04T01:32:57.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29472025-06-04T01:32:57.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29482025-06-04T01:32:57.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29492025-06-04T01:32:57.750ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29502025-06-04T01:32:57.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29512025-06-04T01:32:57.778ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29522025-06-04T01:32:57.778ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
29532025-06-04T01:32:57.778ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29542025-06-04T01:32:57.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29552025-06-04T01:32:57.779ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29562025-06-04T01:32:57.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29572025-06-04T01:32:57.781ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29582025-06-04T01:32:57.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29592025-06-04T01:32:57.782ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
29602025-06-04T01:32:57.782ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29612025-06-04T01:32:57.782ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fd4c9bd-e4ed-4c50-9774-dbc806e7d6ce", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
29622025-06-04T01:32:57.782ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29632025-06-04T01:32:57.784ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29642025-06-04T01:32:57.787ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
29652025-06-04T01:32:57.787ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 54.51159ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
29662025-06-04T01:32:57.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29672025-06-04T01:32:57.792ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29682025-06-04T01:32:57.792ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29692025-06-04T01:32:57.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29702025-06-04T01:32:57.793ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 59.648905ms iteration = 30 status = {"errors":[],"requests_created_ok":[]}
29712025-06-04T01:32:57.794ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
29722025-06-04T01:32:57.794ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29732025-06-04T01:32:57.794ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd5bebd8-c926-4434-9bb6-cfc6987294ae", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
29742025-06-04T01:32:57.794ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29752025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29762025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29772025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29782025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29792025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29802025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29812025-06-04T01:32:57.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29822025-06-04T01:32:57.797ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29832025-06-04T01:32:57.797ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 63.232515ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
29842025-06-04T01:32:57.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29852025-06-04T01:32:57.797ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29862025-06-04T01:32:57.797ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29872025-06-04T01:32:57.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29882025-06-04T01:32:57.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29892025-06-04T01:32:57.798ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29902025-06-04T01:32:57.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29912025-06-04T01:32:57.798ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1914, tv_nsec: 893092593 } })
29922025-06-04T01:32:57.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
29932025-06-04T01:32:57.798ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
29942025-06-04T01:32:57.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29952025-06-04T01:32:57.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29962025-06-04T01:32:57.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29972025-06-04T01:32:57.800ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29982025-06-04T01:32:57.800ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.138926ms iteration = 30 status = {"errors":[],"finish_invoked_ok":[]}
29992025-06-04T01:32:57.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30002025-06-04T01:32:57.802ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012025-06-04T01:32:57.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30022025-06-04T01:32:57.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30032025-06-04T01:32:57.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30042025-06-04T01:32:57.803ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30052025-06-04T01:32:57.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30062025-06-04T01:32:57.818ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
30072025-06-04T01:32:57.818ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 86.573491ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
30082025-06-04T01:32:57.828ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30092025-06-04T01:32:57.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30102025-06-04T01:32:57.835ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 101.574641ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30112025-06-04T01:32:57.837ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 104.480663ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30122025-06-04T01:32:57.839ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30132025-06-04T01:32:57.839ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30142025-06-04T01:32:57.839ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
30152025-06-04T01:32:57.839ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30162025-06-04T01:32:57.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30172025-06-04T01:32:57.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30182025-06-04T01:32:57.840ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 108.080514ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30192025-06-04T01:32:57.844ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30202025-06-04T01:32:57.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30212025-06-04T01:32:57.847ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30222025-06-04T01:32:57.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30232025-06-04T01:32:57.861ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30242025-06-04T01:32:57.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30252025-06-04T01:32:57.864ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 132.681234ms iteration = 30 status = null
30262025-06-04T01:32:57.866ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30272025-06-04T01:32:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30282025-06-04T01:32:57.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30292025-06-04T01:32:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30302025-06-04T01:32:57.872ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 141.329767ms iteration = 30 status = {}
30312025-06-04T01:32:57.875ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
30322025-06-04T01:32:57.875ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30332025-06-04T01:32:57.875ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6647ca0-0b4a-4712-a788-7348f433f48a", "content-length": "26", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
30342025-06-04T01:32:57.875ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30352025-06-04T01:32:57.876ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30362025-06-04T01:32:57.876ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
30372025-06-04T01:32:57.876ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 143.743525ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
30382025-06-04T01:32:57.876ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
30392025-06-04T01:32:57.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30402025-06-04T01:32:57.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30412025-06-04T01:32:57.884ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 150.68524ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
30422025-06-04T01:32:57.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432025-06-04T01:32:57.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
30442025-06-04T01:32:57.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30452025-06-04T01:32:57.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30462025-06-04T01:32:57.892ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 159.519458ms iteration = 30 status = 0
30472025-06-04T01:32:57.906ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
30482025-06-04T01:32:57.907ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 174.409666ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
30492025-06-04T01:32:57.911ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 179.266582ms iteration = 33 status = {"physical_disks_added":0}
30502025-06-04T01:32:57.950ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30512025-06-04T01:32:57.950ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30522025-06-04T01:32:57.950ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73647d9-666a-4ad3-bd3d-b1f6c5c4ef79", "content-length": "372", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
30532025-06-04T01:32:57.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30542025-06-04T01:32:57.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30552025-06-04T01:32:57.985ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30562025-06-04T01:32:57.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30572025-06-04T01:32:57.987ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582025-06-04T01:32:57.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30592025-06-04T01:32:57.988ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30602025-06-04T01:32:57.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30612025-06-04T01:32:57.990ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30622025-06-04T01:32:57.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30632025-06-04T01:32:57.991ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30642025-06-04T01:32:57.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30652025-06-04T01:32:57.993ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
30662025-06-04T01:32:57.993ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30672025-06-04T01:32:57.993ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "47b2cec2-37ba-4e18-bf3e-b9351d9e51e5", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
30682025-06-04T01:32:57.993ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30692025-06-04T01:32:57.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30702025-06-04T01:32:57.996ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30712025-06-04T01:32:57.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30722025-06-04T01:32:57.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30732025-06-04T01:32:57.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30742025-06-04T01:32:57.999ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30752025-06-04T01:32:57.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30762025-06-04T01:32:58.025ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30772025-06-04T01:32:58.025ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "431e669f-aab0-441b-853d-851c8a9f0c3d", "content-length": "691", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
30782025-06-04T01:32:58.025ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
30792025-06-04T01:32:58.025ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
30802025-06-04T01:32:58.025ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
30812025-06-04T01:32:58.026ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 293.570391ms iteration = 31 status = {}
30822025-06-04T01:32:58.027ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
30832025-06-04T01:32:58.027ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30842025-06-04T01:32:58.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30852025-06-04T01:32:58.029ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30862025-06-04T01:32:58.029ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30872025-06-04T01:32:58.029ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0474b088-fab7-4bd2-9bc8-f0b521b8f940", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
30882025-06-04T01:32:58.030ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 298.266233ms iteration = 31 status = {}
30892025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
30902025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
30912025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
30922025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
30932025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30942025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30952025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30962025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30972025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30982025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30992025-06-04T01:32:58.039ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
31002025-06-04T01:32:58.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31012025-06-04T01:32:58.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31022025-06-04T01:32:58.041ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
31032025-06-04T01:32:58.041ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
31042025-06-04T01:32:58.135ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8ac6197d-dcf9-4752-b0b6-ff1326568b67", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
31052025-06-04T01:32:58.135ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
31062025-06-04T01:32:58.135ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31072025-06-04T01:32:58.135ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1da30749-7496-4eec-9dc5-a46a6cfa93c1", "content-length": "682", "date": "Wed, 04 Jun 2025 01:32:57 GMT"} })
31082025-06-04T01:32:58.135ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31092025-06-04T01:32:58.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31102025-06-04T01:32:58.138ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31112025-06-04T01:32:58.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31122025-06-04T01:32:58.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31132025-06-04T01:32:58.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31142025-06-04T01:32:58.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31152025-06-04T01:32:58.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31162025-06-04T01:32:58.151ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 418.961831ms iteration = 30 status = {}
31172025-06-04T01:32:58.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e3eda3e9-5e85-4839-a194-47a97f080947 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31182025-06-04T01:32:58.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e3eda3e9-5e85-4839-a194-47a97f080947 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31192025-06-04T01:32:58.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = e3eda3e9-5e85-4839-a194-47a97f080947 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31202025-06-04T01:32:59.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 868f64cd-fb2c-4b0f-9908-1efb743ea04e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31212025-06-04T01:32:59.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 868f64cd-fb2c-4b0f-9908-1efb743ea04e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31222025-06-04T01:32:59.205ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 868f64cd-fb2c-4b0f-9908-1efb743ea04e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31232025-06-04T01:33:00.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 284ad572-eefe-4701-8866-24ef4a9af198 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31242025-06-04T01:33:00.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 284ad572-eefe-4701-8866-24ef4a9af198 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31252025-06-04T01:33:00.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2309 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 284ad572-eefe-4701-8866-24ef4a9af198 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31262025-06-04T01:33:00.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 651 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = b1043521-e2ab-4e18-9202-d5a9c3d8c8cc response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
31272025-06-04T01:33:01.236ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d45738d6-809f-420f-a2b6-801081c59006 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31282025-06-04T01:33:01.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d45738d6-809f-420f-a2b6-801081c59006 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31292025-06-04T01:33:01.238ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2866 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d45738d6-809f-420f-a2b6-801081c59006 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31302025-06-04T01:33:02.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd98bf1d-6c18-4c51-a780-2213a3e9dfad roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31312025-06-04T01:33:02.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd98bf1d-6c18-4c51-a780-2213a3e9dfad resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31322025-06-04T01:33:02.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = cd98bf1d-6c18-4c51-a780-2213a3e9dfad response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31332025-06-04T01:33:03.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4773631d-7310-4387-94a4-00f059e6b609 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31342025-06-04T01:33:03.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4773631d-7310-4387-94a4-00f059e6b609 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31352025-06-04T01:33:03.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2895 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4773631d-7310-4387-94a4-00f059e6b609 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31362025-06-04T01:33:04.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 379df9c0-b843-4ce6-8631-02c74f37b788 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31372025-06-04T01:33:04.284ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 379df9c0-b843-4ce6-8631-02c74f37b788 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31382025-06-04T01:33:04.286ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 379df9c0-b843-4ce6-8631-02c74f37b788 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31392025-06-04T01:33:05.299ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00a0c9a6-3577-4d4a-82ab-15cf0c06915b roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31402025-06-04T01:33:05.300ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00a0c9a6-3577-4d4a-82ab-15cf0c06915b resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31412025-06-04T01:33:05.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 00a0c9a6-3577-4d4a-82ab-15cf0c06915b response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31422025-06-04T01:33:06.315ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4a47b0fa-cba3-4c59-afad-7a0d3502c888 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31432025-06-04T01:33:06.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4a47b0fa-cba3-4c59-afad-7a0d3502c888 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31442025-06-04T01:33:06.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2502 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4a47b0fa-cba3-4c59-afad-7a0d3502c888 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31452025-06-04T01:33:07.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6e5a74fb-a05b-49de-a2cd-1021800eaa02 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31462025-06-04T01:33:07.331ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6e5a74fb-a05b-49de-a2cd-1021800eaa02 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31472025-06-04T01:33:07.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2876 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 6e5a74fb-a05b-49de-a2cd-1021800eaa02 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31482025-06-04T01:33:07.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
31492025-06-04T01:33:07.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31502025-06-04T01:33:07.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31512025-06-04T01:33:07.732ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
31522025-06-04T01:33:07.732ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31532025-06-04T01:33:07.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31542025-06-04T01:33:07.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31552025-06-04T01:33:07.766ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31562025-06-04T01:33:07.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31572025-06-04T01:33:07.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31582025-06-04T01:33:07.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31592025-06-04T01:33:07.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31602025-06-04T01:33:07.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31612025-06-04T01:33:07.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31622025-06-04T01:33:07.799ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.921008ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31632025-06-04T01:33:07.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31642025-06-04T01:33:07.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
31652025-06-04T01:33:07.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31662025-06-04T01:33:07.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31672025-06-04T01:33:07.814ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
31682025-06-04T01:33:07.814ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 82.732463ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
31692025-06-04T01:33:08.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2aa9125f-44f0-49df-a73b-8e856febf9bc roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31702025-06-04T01:33:08.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2aa9125f-44f0-49df-a73b-8e856febf9bc resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31712025-06-04T01:33:08.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2aa9125f-44f0-49df-a73b-8e856febf9bc response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31722025-06-04T01:33:09.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = febde5a3-6338-4911-86e7-981b385ae542 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31732025-06-04T01:33:09.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = febde5a3-6338-4911-86e7-981b385ae542 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31742025-06-04T01:33:09.366ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2704 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = febde5a3-6338-4911-86e7-981b385ae542 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31752025-06-04T01:33:10.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a89762ad-eec5-4075-9cb3-0752508e68a4 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31762025-06-04T01:33:10.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a89762ad-eec5-4075-9cb3-0752508e68a4 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31772025-06-04T01:33:10.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2510 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = a89762ad-eec5-4075-9cb3-0752508e68a4 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31782025-06-04T01:33:10.458ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 514 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 39559406-39ea-451b-99fc-b6caaa17d4b7 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
31792025-06-04T01:33:11.396ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b139a50e-9f1b-40e2-856b-80e05b956b1a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31802025-06-04T01:33:11.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b139a50e-9f1b-40e2-856b-80e05b956b1a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31812025-06-04T01:33:11.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b139a50e-9f1b-40e2-856b-80e05b956b1a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31822025-06-04T01:33:12.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e74fdbe-1c80-49d8-a9e6-3219c4f45c35 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31832025-06-04T01:33:12.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e74fdbe-1c80-49d8-a9e6-3219c4f45c35 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31842025-06-04T01:33:12.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2352 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7e74fdbe-1c80-49d8-a9e6-3219c4f45c35 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31852025-06-04T01:33:13.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1c7785d-1b0d-4f09-8b6f-281eaf82d3aa roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31862025-06-04T01:33:13.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1c7785d-1b0d-4f09-8b6f-281eaf82d3aa resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31872025-06-04T01:33:13.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2221 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = f1c7785d-1b0d-4f09-8b6f-281eaf82d3aa response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31882025-06-04T01:33:14.442ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2161eb6a-27f1-4925-bae1-2b739a4c1ada roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31892025-06-04T01:33:14.442ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2161eb6a-27f1-4925-bae1-2b739a4c1ada resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31902025-06-04T01:33:14.444ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2217 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 2161eb6a-27f1-4925-bae1-2b739a4c1ada response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31912025-06-04T01:33:15.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ebf2779f-0b2d-40de-b5f4-1384c3cecb1d roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31922025-06-04T01:33:15.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ebf2779f-0b2d-40de-b5f4-1384c3cecb1d resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31932025-06-04T01:33:15.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2372 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = ebf2779f-0b2d-40de-b5f4-1384c3cecb1d response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31942025-06-04T01:33:16.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b69047b1-9326-46db-841c-7131d1014386 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31952025-06-04T01:33:16.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b69047b1-9326-46db-841c-7131d1014386 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31962025-06-04T01:33:16.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2158 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b69047b1-9326-46db-841c-7131d1014386 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31972025-06-04T01:33:17.488ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7ff04ced-5d8d-46fe-ac4f-0e4bb3ce1610 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31982025-06-04T01:33:17.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7ff04ced-5d8d-46fe-ac4f-0e4bb3ce1610 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
31992025-06-04T01:33:17.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2348 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7ff04ced-5d8d-46fe-ac4f-0e4bb3ce1610 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32002025-06-04T01:33:17.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
32012025-06-04T01:33:17.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32022025-06-04T01:33:17.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32032025-06-04T01:33:17.731ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
32042025-06-04T01:33:17.731ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32052025-06-04T01:33:17.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32062025-06-04T01:33:17.734ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32072025-06-04T01:33:17.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32082025-06-04T01:33:17.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32092025-06-04T01:33:17.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.666113ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32102025-06-04T01:33:17.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32112025-06-04T01:33:17.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32122025-06-04T01:33:17.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32132025-06-04T01:33:17.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32142025-06-04T01:33:17.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32152025-06-04T01:33:17.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32162025-06-04T01:33:17.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
32172025-06-04T01:33:17.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32182025-06-04T01:33:17.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32192025-06-04T01:33:17.787ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
32202025-06-04T01:33:17.787ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.29051ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
32212025-06-04T01:33:18.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1e93aea8-0d5e-4073-ad72-0dd03910d5a7 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32222025-06-04T01:33:18.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1e93aea8-0d5e-4073-ad72-0dd03910d5a7 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32232025-06-04T01:33:18.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2478 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 1e93aea8-0d5e-4073-ad72-0dd03910d5a7 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32242025-06-04T01:33:19.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a5d310-1a6f-4235-b24a-6e1e7f4d10c2 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32252025-06-04T01:33:19.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a5d310-1a6f-4235-b24a-6e1e7f4d10c2 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32262025-06-04T01:33:19.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2326 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 63a5d310-1a6f-4235-b24a-6e1e7f4d10c2 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32272025-06-04T01:33:20.459ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 934 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 1bdcd58e-e832-44ab-aecd-629ed136c0a7 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
32282025-06-04T01:33:20.535ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd654a17-5b36-4155-bbb2-0c05313c297a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32292025-06-04T01:33:20.536ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd654a17-5b36-4155-bbb2-0c05313c297a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32302025-06-04T01:33:20.538ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2722 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = dd654a17-5b36-4155-bbb2-0c05313c297a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32312025-06-04T01:33:21.551ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 214dc66d-26e6-4e39-a6f6-e2f52b23bb8a roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32322025-06-04T01:33:21.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 214dc66d-26e6-4e39-a6f6-e2f52b23bb8a resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32332025-06-04T01:33:21.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2239 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 214dc66d-26e6-4e39-a6f6-e2f52b23bb8a response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32342025-06-04T01:33:22.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7619d85a-7030-4a93-88af-93c22a1345c1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32352025-06-04T01:33:22.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7619d85a-7030-4a93-88af-93c22a1345c1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32362025-06-04T01:33:22.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 7619d85a-7030-4a93-88af-93c22a1345c1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32372025-06-04T01:33:23.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 43ab14c6-786e-42f1-ba17-62dd1b27b8dd roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32382025-06-04T01:33:23.581ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 43ab14c6-786e-42f1-ba17-62dd1b27b8dd resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32392025-06-04T01:33:23.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2715 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 43ab14c6-786e-42f1-ba17-62dd1b27b8dd response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32402025-06-04T01:33:24.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8b301d53-22bc-46c6-88ba-8dd229d7dfec roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32412025-06-04T01:33:24.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8b301d53-22bc-46c6-88ba-8dd229d7dfec resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32422025-06-04T01:33:24.599ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2134 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 8b301d53-22bc-46c6-88ba-8dd229d7dfec response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32432025-06-04T01:33:25.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f8f806c-8d44-4234-84f3-46f2a9336fc9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32442025-06-04T01:33:25.614ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f8f806c-8d44-4234-84f3-46f2a9336fc9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32452025-06-04T01:33:25.615ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 4f8f806c-8d44-4234-84f3-46f2a9336fc9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32462025-06-04T01:33:26.630ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c9ffdded-a9c9-4fd1-a607-a25e9bfcf0ef roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32472025-06-04T01:33:26.631ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c9ffdded-a9c9-4fd1-a607-a25e9bfcf0ef resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32482025-06-04T01:33:26.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2514 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c9ffdded-a9c9-4fd1-a607-a25e9bfcf0ef response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32492025-06-04T01:33:27.646ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5879288e-588d-43e1-888c-a237164c2e7c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32502025-06-04T01:33:27.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5879288e-588d-43e1-888c-a237164c2e7c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32512025-06-04T01:33:27.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2488 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 5879288e-588d-43e1-888c-a237164c2e7c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
32522025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
32532025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
32542025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
32552025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32562025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32572025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
32582025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
32592025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32602025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
32612025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
32622025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
32632025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
32642025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
32652025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
32662025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32672025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
32682025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
32692025-06-04T01:33:27.731ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-04 01:23:27.731409383 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
32702025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
32712025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
32722025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
32732025-06-04T01:33:27.731ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 380e7fae-5d5a-479a-b890-8108906f1718
32742025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32752025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
32762025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.531\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"380e7fae-5d5a-479a-b890-8108906f1718"}
32772025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
32782025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32792025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
32802025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32812025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
32822025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 7 reason = Timeout
32832025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
32842025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
32852025-06-04T01:33:27.731ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
32862025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
32872025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32882025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
32892025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
32902025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
32912025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32922025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
32932025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32942025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
32952025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
32962025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32972025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32982025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
32992025-06-04T01:33:27.731ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33002025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33012025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
33022025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33032025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
33042025-06-04T01:33:27.731ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
33052025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
33062025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
33072025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 32 reason = Timeout
33082025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33092025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33102025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33112025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33122025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33132025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
33142025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33152025-06-04T01:33:27.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33162025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
33172025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33182025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33192025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33202025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
33212025-06-04T01:33:27.732ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2d727093-e092-497f-a1e2-dd76c1241db0.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f6bcd2c3-045d-412e-b32f-07c614adbb05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1944, tv_nsec: 826946766 } })
33222025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33232025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
33242025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33252025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33262025-06-04T01:33:27.732ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4cce14d3-021e-4041-915d-88da72189266.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("657dd1d3-7198-439e-842d-83bce056de40.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aacbc88f-98f9-423b-9e93-03d859fe410d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1944, tv_nsec: 826996777 } })
33272025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33282025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33292025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33302025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33312025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33322025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33332025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33342025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
33352025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33362025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
33372025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
33382025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33392025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33402025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
33412025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.155192ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
33422025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
33432025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33442025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33452025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
33462025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.358568ms iteration = 16 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
33472025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33482025-06-04T01:33:27.732ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33492025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33502025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
33512025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 16 reason = Timeout
33522025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33532025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
33542025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
33552025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
33562025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 4 reason = Timeout
33572025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
33582025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 16 reason = Timeout
33592025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
33602025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
33612025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
33622025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 31 reason = Timeout
33632025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
33642025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
33652025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
33662025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
33672025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33682025-06-04T01:33:27.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
33692025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33702025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
33712025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
33722025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
33732025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33742025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33752025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
33762025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
33772025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
33782025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33792025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33802025-06-04T01:33:27.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33812025-06-04T01:33:27.734ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1944, tv_nsec: 829559039 } })
33822025-06-04T01:33:27.735ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
33832025-06-04T01:33:27.735ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-04 01:23:27.731409383 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
33842025-06-04T01:33:27.735ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.391323ms iteration = 16 status = {"expiration":"2025-06-04T01:23:27.731409383Z","pruned":[]}
33852025-06-04T01:33:27.735ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.597079ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
33862025-06-04T01:33:27.737ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33872025-06-04T01:33:27.737ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1944, tv_nsec: 832156002 } })
33882025-06-04T01:33:27.737ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
33892025-06-04T01:33:27.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33902025-06-04T01:33:27.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33912025-06-04T01:33:27.740ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33922025-06-04T01:33:27.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33932025-06-04T01:33:27.741ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
33942025-06-04T01:33:27.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
33952025-06-04T01:33:27.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
33962025-06-04T01:33:27.744ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33972025-06-04T01:33:27.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33982025-06-04T01:33:27.745ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33992025-06-04T01:33:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34002025-06-04T01:33:27.748ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34012025-06-04T01:33:27.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34022025-06-04T01:33:27.750ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34032025-06-04T01:33:27.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34042025-06-04T01:33:27.753ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34052025-06-04T01:33:27.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34062025-06-04T01:33:27.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34072025-06-04T01:33:27.782ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34082025-06-04T01:33:27.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34092025-06-04T01:33:27.783ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34102025-06-04T01:33:27.784ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34112025-06-04T01:33:27.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34122025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34132025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34142025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34152025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
34162025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
34172025-06-04T01:33:27.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
34182025-06-04T01:33:27.786ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34192025-06-04T01:33:27.786ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
34202025-06-04T01:33:27.786ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34212025-06-04T01:33:27.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34222025-06-04T01:33:27.787ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34232025-06-04T01:33:27.787ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34242025-06-04T01:33:27.788ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
34252025-06-04T01:33:27.788ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
34262025-06-04T01:33:27.788ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.010652ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
34272025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 55.723034ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
34282025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
34292025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
34302025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
34312025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34322025-06-04T01:33:27.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34332025-06-04T01:33:27.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34342025-06-04T01:33:27.791ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34352025-06-04T01:33:27.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34362025-06-04T01:33:27.792ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
34372025-06-04T01:33:27.792ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 58.638966ms iteration = 16 status = {"by_rx":{},"error":null}
34382025-06-04T01:33:27.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34392025-06-04T01:33:27.792ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34402025-06-04T01:33:27.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34412025-06-04T01:33:27.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34422025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34432025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34442025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34452025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34462025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34472025-06-04T01:33:27.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34482025-06-04T01:33:27.796ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34492025-06-04T01:33:27.796ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34502025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34512025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34522025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34532025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34542025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34552025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34562025-06-04T01:33:27.798ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34572025-06-04T01:33:27.799ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34582025-06-04T01:33:27.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34592025-06-04T01:33:27.800ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34602025-06-04T01:33:27.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34612025-06-04T01:33:27.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34622025-06-04T01:33:27.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34632025-06-04T01:33:27.801ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34642025-06-04T01:33:27.802ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34652025-06-04T01:33:27.802ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
34662025-06-04T01:33:27.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34672025-06-04T01:33:27.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34682025-06-04T01:33:27.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34692025-06-04T01:33:27.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
34702025-06-04T01:33:27.803ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34712025-06-04T01:33:27.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34722025-06-04T01:33:27.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34732025-06-04T01:33:27.803ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34742025-06-04T01:33:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34752025-06-04T01:33:27.804ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34762025-06-04T01:33:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34772025-06-04T01:33:27.804ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
34782025-06-04T01:33:27.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
34792025-06-04T01:33:27.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
34802025-06-04T01:33:27.805ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
34812025-06-04T01:33:27.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
34822025-06-04T01:33:27.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34832025-06-04T01:33:27.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34842025-06-04T01:33:27.815ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34852025-06-04T01:33:27.817ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
34862025-06-04T01:33:27.817ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
34872025-06-04T01:33:27.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
34882025-06-04T01:33:27.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34892025-06-04T01:33:27.820ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-4adf0198-0a8a-47c8-a668-dc22afd01e24.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1944, tv_nsec: 914833853 } })
34902025-06-04T01:33:27.820ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
34912025-06-04T01:33:27.820ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34922025-06-04T01:33:27.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34932025-06-04T01:33:27.822ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34942025-06-04T01:33:27.822ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34952025-06-04T01:33:27.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34962025-06-04T01:33:27.823ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34972025-06-04T01:33:27.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34982025-06-04T01:33:27.829ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34992025-06-04T01:33:27.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35002025-06-04T01:33:27.833ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 99.736649ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
35012025-06-04T01:33:27.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35022025-06-04T01:33:27.834ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 102.859627ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35032025-06-04T01:33:27.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
35042025-06-04T01:33:27.837ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35052025-06-04T01:33:27.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35062025-06-04T01:33:27.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35072025-06-04T01:33:27.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35082025-06-04T01:33:27.840ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35092025-06-04T01:33:27.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35102025-06-04T01:33:27.844ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 110.316597ms iteration = 16 status = {"dispatched":[],"errors":[],"glob_version":"146.0.0","globs_reprocessed":{},"no_receivers":[]}
35112025-06-04T01:33:27.844ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 112.664042ms iteration = 7 status = {"nsuccess":5}
35122025-06-04T01:33:27.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35132025-06-04T01:33:27.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35142025-06-04T01:33:27.847ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 113.873687ms iteration = 31 status = {"errors":[],"finish_invoked_ok":[]}
35152025-06-04T01:33:27.849ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35162025-06-04T01:33:27.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35172025-06-04T01:33:27.852ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 118.463885ms iteration = 31 status = {"errors":[],"requests_created_ok":[]}
35182025-06-04T01:33:27.854ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35192025-06-04T01:33:27.854ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35202025-06-04T01:33:27.854ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35212025-06-04T01:33:27.854ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35222025-06-04T01:33:27.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35232025-06-04T01:33:27.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35242025-06-04T01:33:27.863ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
35252025-06-04T01:33:27.863ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
35262025-06-04T01:33:27.863ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
35272025-06-04T01:33:27.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35282025-06-04T01:33:27.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35292025-06-04T01:33:27.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35302025-06-04T01:33:27.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35312025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35322025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35332025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35342025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35352025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35362025-06-04T01:33:27.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35372025-06-04T01:33:27.871ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
35382025-06-04T01:33:27.871ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
35392025-06-04T01:33:27.871ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 139.880467ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
35402025-06-04T01:33:27.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35412025-06-04T01:33:27.871ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35422025-06-04T01:33:27.871ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 140.36316ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35432025-06-04T01:33:27.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35442025-06-04T01:33:27.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35452025-06-04T01:33:27.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 141.696147ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35462025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
35472025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35482025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35492025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35502025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35512025-06-04T01:33:27.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35522025-06-04T01:33:27.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35532025-06-04T01:33:27.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35542025-06-04T01:33:27.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35552025-06-04T01:33:27.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35562025-06-04T01:33:27.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35572025-06-04T01:33:27.878ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35582025-06-04T01:33:27.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35592025-06-04T01:33:27.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35602025-06-04T01:33:27.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35612025-06-04T01:33:27.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35622025-06-04T01:33:27.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35632025-06-04T01:33:27.883ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
35642025-06-04T01:33:27.883ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
35652025-06-04T01:33:27.883ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
35662025-06-04T01:33:27.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35672025-06-04T01:33:27.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35682025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
35692025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35702025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35712025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35722025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
35732025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35742025-06-04T01:33:27.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35752025-06-04T01:33:27.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35762025-06-04T01:33:27.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35772025-06-04T01:33:27.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35782025-06-04T01:33:27.886ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
35792025-06-04T01:33:27.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
35802025-06-04T01:33:27.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
35812025-06-04T01:33:27.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35822025-06-04T01:33:27.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35832025-06-04T01:33:27.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35842025-06-04T01:33:27.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35852025-06-04T01:33:27.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35862025-06-04T01:33:27.897ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35872025-06-04T01:33:27.897ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35882025-06-04T01:33:27.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35892025-06-04T01:33:27.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35902025-06-04T01:33:27.897ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
35912025-06-04T01:33:27.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35922025-06-04T01:33:27.900ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 169.19361ms iteration = 31 status = 0
35932025-06-04T01:33:27.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35942025-06-04T01:33:27.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35952025-06-04T01:33:27.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35962025-06-04T01:33:27.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35972025-06-04T01:33:27.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35982025-06-04T01:33:27.905ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35992025-06-04T01:33:27.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36002025-06-04T01:33:27.905ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 172.118962ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
36012025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
36022025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36032025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36042025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
36052025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
36062025-06-04T01:33:27.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
36072025-06-04T01:33:27.908ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36082025-06-04T01:33:27.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36092025-06-04T01:33:27.910ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
36102025-06-04T01:33:27.910ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36112025-06-04T01:33:27.910ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "526f99cc-bcde-48af-af75-adb2015fea2d", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:27 GMT"} })
36122025-06-04T01:33:27.910ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36132025-06-04T01:33:27.910ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
36142025-06-04T01:33:27.910ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 179.354105ms iteration = 7 status = {"blueprint_id":"380e7fae-5d5a-479a-b890-8108906f1718","inventory_collection_id":"38598be5-1af3-47c9-8fd1-35a15c522a32","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
36152025-06-04T01:33:27.912ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36162025-06-04T01:33:27.913ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36172025-06-04T01:33:27.913ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36182025-06-04T01:33:27.913ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 181.551196ms iteration = 32 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36192025-06-04T01:33:27.923ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
36202025-06-04T01:33:27.923ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 192.35471ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
36212025-06-04T01:33:27.923ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
36222025-06-04T01:33:27.923ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 0 version = 2
36232025-06-04T01:33:27.923ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36242025-06-04T01:33:27.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36252025-06-04T01:33:27.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36262025-06-04T01:33:27.928ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36272025-06-04T01:33:27.928ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36282025-06-04T01:33:27.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36292025-06-04T01:33:27.931ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d6e326ca-ba5c-41c0-bb87-fc383db58ae1 dns_zone_name = oxide.test found_so_far = 4 version = 2
36302025-06-04T01:33:27.931ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
36312025-06-04T01:33:27.931ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-04 01:18:52.185847 UTC dns_group = external generation = 2
36322025-06-04T01:33:27.931ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 200.177509ms iteration = 16 status = {"generation":2}
36332025-06-04T01:33:27.938ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
36342025-06-04T01:33:27.938ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 206.574509ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
36352025-06-04T01:33:27.956ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36362025-06-04T01:33:27.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
36372025-06-04T01:33:27.960ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36382025-06-04T01:33:27.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36392025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
36402025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
36412025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36422025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36432025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-04T01:18:23.828702Z, time_modified: 2025-06-04T01:18:23.828702Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, db_silo: Silo { identity: SiloIdentity { id: a645d67d-1f6e-4ba4-9148-3bd6c69e26d4, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-04T01:18:52.140088Z, time_modified: 2025-06-04T01:18:52.140088Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916" }] }) }
36442025-06-04T01:33:27.962ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 231.456967ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]}},"default_endpoint":{"silo_id":"a645d67d-1f6e-4ba4-9148-3bd6c69e26d4","tls_certs":["84374be50f40b58ffe85f3d329327d5430ba4a2a60018c6cd856cb4b47a1a916"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
36452025-06-04T01:33:27.965ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36462025-06-04T01:33:27.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36472025-06-04T01:33:27.967ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 233.537136ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
36482025-06-04T01:33:27.968ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 236.813927ms iteration = 34 status = {"physical_disks_added":0}
36492025-06-04T01:33:27.977ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36502025-06-04T01:33:27.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36512025-06-04T01:33:28.017ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
36522025-06-04T01:33:28.033ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d zone_address = fd00:1122:3344:101::2
36532025-06-04T01:33:28.033ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36542025-06-04T01:33:28.033ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a0b7e6a-a800-4367-9b8f-f249d3d265b9", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:27 GMT"} })
36552025-06-04T01:33:28.033ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36562025-06-04T01:33:28.035ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
36572025-06-04T01:33:28.055ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
36582025-06-04T01:33:28.055ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36592025-06-04T01:33:28.055ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bd36095-a294-4f72-8d4a-c02f502536df", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:27 GMT"} })
36602025-06-04T01:33:28.055ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36612025-06-04T01:33:28.057ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36622025-06-04T01:33:28.059ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
36632025-06-04T01:33:28.059ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 327.498623ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
36642025-06-04T01:33:28.091ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
36652025-06-04T01:33:28.138ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 407.272403ms iteration = 31 status = {}
36662025-06-04T01:33:28.161ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36672025-06-04T01:33:28.161ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36682025-06-04T01:33:28.162ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a05b8061-735e-4e25-a556-c50f857c882b", "content-length": "372", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
36692025-06-04T01:33:28.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36702025-06-04T01:33:28.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36712025-06-04T01:33:28.164ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36722025-06-04T01:33:28.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36732025-06-04T01:33:28.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36742025-06-04T01:33:28.166ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36752025-06-04T01:33:28.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36762025-06-04T01:33:28.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36772025-06-04T01:33:28.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36782025-06-04T01:33:28.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36792025-06-04T01:33:28.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36802025-06-04T01:33:28.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
36812025-06-04T01:33:28.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36822025-06-04T01:33:28.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36832025-06-04T01:33:28.208ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
36842025-06-04T01:33:28.208ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
36852025-06-04T01:33:28.208ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2f264b0e-c10e-4dec-bbf1-8d0ae865bb99", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
36862025-06-04T01:33:28.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36872025-06-04T01:33:28.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36882025-06-04T01:33:28.240ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36892025-06-04T01:33:28.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36902025-06-04T01:33:28.257ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
36912025-06-04T01:33:28.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "63518a84-d59f-4e13-93c2-d4073d05f861", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36922025-06-04T01:33:28.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36932025-06-04T01:33:28.258ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
36942025-06-04T01:33:28.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
36952025-06-04T01:33:28.260ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "3ddcd69f-161e-423f-992d-02dcf2dbe2fb", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36962025-06-04T01:33:28.260ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36972025-06-04T01:33:28.260ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36982025-06-04T01:33:28.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36992025-06-04T01:33:28.261ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37002025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = a504afbb-531f-4cf3-ad39-de66d383349e dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
37012025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
37022025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-04 01:18:51.716768 UTC dns_group = internal generation = 1
37032025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 531.065517ms iteration = 16 status = {"generation":1}
37042025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37052025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5fd78899-89c9-4d16-970c-ebfb87d59208", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
37062025-06-04T01:33:28.262ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
37072025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 531.313004ms iteration = 31 status = null
37082025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37092025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37102025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37112025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37122025-06-04T01:33:28.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37132025-06-04T01:33:28.263ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37142025-06-04T01:33:28.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37152025-06-04T01:33:28.264ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
37162025-06-04T01:33:28.264ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 533.130835ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
37172025-06-04T01:33:28.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
37182025-06-04T01:33:28.264ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
37192025-06-04T01:33:28.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
37202025-06-04T01:33:28.267ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
37212025-06-04T01:33:28.267ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
37222025-06-04T01:33:28.267ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b9088535-84c4-45a3-a77b-c5c775443d8d", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
37232025-06-04T01:33:28.267ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
37242025-06-04T01:33:28.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
37252025-06-04T01:33:28.268ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37262025-06-04T01:33:28.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37272025-06-04T01:33:28.269ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37282025-06-04T01:33:28.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "46b65082-ba38-484a-9cfc-b12bc2b8a525", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
37292025-06-04T01:33:28.269ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
37302025-06-04T01:33:28.270ZINFOnexus (BackgroundTasks): VPC route manager sled 4adf0198-0a8a-47c8-a668-dc22afd01e24 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
37312025-06-04T01:33:28.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
37322025-06-04T01:33:28.270ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
37332025-06-04T01:33:28.270ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37342025-06-04T01:33:28.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
37352025-06-04T01:33:28.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 539.628117ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
37362025-06-04T01:33:28.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
37372025-06-04T01:33:28.271ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7f6f422f-a6c9-4a73-a7cc-3de17477b46b", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
37382025-06-04T01:33:28.273ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
37392025-06-04T01:33:28.273ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 541.585782ms iteration = 32 status = {}
37402025-06-04T01:33:28.274ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "a8a541ce-bcc0-4555-af11-f13af3cbc3b6", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
37412025-06-04T01:33:28.274ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
37422025-06-04T01:33:28.274ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "b70bfacb-6faa-4ec2-86d6-64365add04cf", "content-length": "26", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
37432025-06-04T01:33:28.274ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
37442025-06-04T01:33:28.276ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 542.690133ms iteration = 4 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:23:28.289445677Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:23:28.289759036Z"},{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:28:28.373965795Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:28:28.374676266Z"},{"generation":1,"operation":"put_config","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:33:28.274140918Z"},{"operation":"list","target_sled":"4adf0198-0a8a-47c8-a668-dc22afd01e24","time":"2025-06-04T01:33:28.274315972Z"}]}
37452025-06-04T01:33:28.296ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37462025-06-04T01:33:28.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37472025-06-04T01:33:28.297ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37482025-06-04T01:33:28.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
37492025-06-04T01:33:28.300ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37502025-06-04T01:33:28.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37512025-06-04T01:33:28.302ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37522025-06-04T01:33:28.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37532025-06-04T01:33:28.304ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37542025-06-04T01:33:28.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37552025-06-04T01:33:28.306ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37562025-06-04T01:33:28.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37572025-06-04T01:33:28.308ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37582025-06-04T01:33:28.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37592025-06-04T01:33:28.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37602025-06-04T01:33:28.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37612025-06-04T01:33:28.316ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 94201b00-826a-4b85-8819-6bd7df68fb9a, rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(fbdcc015-16a1-40ce-9a49-d4bde04ee043) } rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
37622025-06-04T01:33:28.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37632025-06-04T01:33:28.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37642025-06-04T01:33:28.328ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
37652025-06-04T01:33:28.328ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = fbdcc015-16a1-40ce-9a49-d4bde04ee043
37662025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
37672025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
37682025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37692025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37702025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37712025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37722025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
37732025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
37742025-06-04T01:33:28.329ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d
37752025-06-04T01:33:28.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37762025-06-04T01:33:28.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37772025-06-04T01:33:28.348ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37782025-06-04T01:33:28.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37792025-06-04T01:33:28.358ZDEBGnexus (BackgroundTasks): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
37802025-06-04T01:33:28.359ZDEBGnexus (BackgroundTasks): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cf576c7-4763-4ea3-a27c-7883ff303e79", "content-length": "691", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
37812025-06-04T01:33:28.359ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
37822025-06-04T01:33:28.359ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37832025-06-04T01:33:28.359ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
37842025-06-04T01:33:28.360ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37852025-06-04T01:33:28.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37862025-06-04T01:33:28.360ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch_location = Switch0
37872025-06-04T01:33:28.361ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
37882025-06-04T01:33:28.361ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 629.614413ms iteration = 32 status = {}
37892025-06-04T01:33:28.362ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37902025-06-04T01:33:28.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
37912025-06-04T01:33:28.366ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37922025-06-04T01:33:28.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37932025-06-04T01:33:28.368ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37942025-06-04T01:33:28.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37952025-06-04T01:33:28.370ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37962025-06-04T01:33:28.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37972025-06-04T01:33:28.371ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37982025-06-04T01:33:28.375ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
37992025-06-04T01:33:28.375ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38002025-06-04T01:33:28.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38012025-06-04T01:33:28.379ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38022025-06-04T01:33:28.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38032025-06-04T01:33:28.381ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38042025-06-04T01:33:28.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38052025-06-04T01:33:28.382ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38062025-06-04T01:33:28.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38072025-06-04T01:33:28.385ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38082025-06-04T01:33:28.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38092025-06-04T01:33:28.387ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38102025-06-04T01:33:28.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38112025-06-04T01:33:28.388ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38122025-06-04T01:33:28.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38132025-06-04T01:33:28.390ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38142025-06-04T01:33:28.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38152025-06-04T01:33:28.391ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38162025-06-04T01:33:28.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
38172025-06-04T01:33:28.395ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38182025-06-04T01:33:28.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38192025-06-04T01:33:28.396ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38202025-06-04T01:33:28.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38212025-06-04T01:33:28.398ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38222025-06-04T01:33:28.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38232025-06-04T01:33:28.399ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38242025-06-04T01:33:28.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
38252025-06-04T01:33:28.403ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38262025-06-04T01:33:28.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38272025-06-04T01:33:28.409ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 14b441f3-45c6-4d21-ba32-0c8a69c8e315, kind: Service { id: 7716fd65-e841-4c70-8dfc-958c8d9211a5 }, name: Name("nexus-7716fd65-e841-4c70-8dfc-958c8d9211a5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 10])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72192272-d1eb-4f52-9910-b5b8aafb4794, kind: Service { id: c0229563-2086-4bd0-82c1-da8e271b757f }, name: Name("nexus-c0229563-2086-4bd0-82c1-da8e271b757f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 195, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bcf19bf5-acd4-44fd-9779-d38eb96f2b74, kind: Service { id: 34942533-fcec-49f1-bc91-3bae11367eac }, name: Name("nexus-34942533-fcec-49f1-bc91-3bae11367eac"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 56513233-0589-4f1f-bfd3-458a6b6602bf, kind: Service { id: 2d727093-e092-497f-a1e2-dd76c1241db0 }, name: Name("external-dns-2d727093-e092-497f-a1e2-dd76c1241db0"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 52])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cf91204e-797e-40be-ad63-a6e7a9ec29d5, kind: Service { id: f6bcd2c3-045d-412e-b32f-07c614adbb05 }, name: Name("external-dns-f6bcd2c3-045d-412e-b32f-07c614adbb05"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 176])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
38282025-06-04T01:33:28.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38292025-06-04T01:33:28.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38302025-06-04T01:33:28.410ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38312025-06-04T01:33:28.412ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
38322025-06-04T01:33:28.412ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38332025-06-04T01:33:28.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38342025-06-04T01:33:28.413ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
38352025-06-04T01:33:28.413ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 56513233-0589-4f1f-bfd3-458a6b6602bf, kind: Service { id: 2d727093-e092-497f-a1e2-dd76c1241db0 }, name: Name("external-dns-2d727093-e092-497f-a1e2-dd76c1241db0"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 52])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cf91204e-797e-40be-ad63-a6e7a9ec29d5, kind: Service { id: f6bcd2c3-045d-412e-b32f-07c614adbb05 }, name: Name("external-dns-f6bcd2c3-045d-412e-b32f-07c614adbb05"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 176])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 14b441f3-45c6-4d21-ba32-0c8a69c8e315, kind: Service { id: 7716fd65-e841-4c70-8dfc-958c8d9211a5 }, name: Name("nexus-7716fd65-e841-4c70-8dfc-958c8d9211a5"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 10])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72192272-d1eb-4f52-9910-b5b8aafb4794, kind: Service { id: c0229563-2086-4bd0-82c1-da8e271b757f }, name: Name("nexus-c0229563-2086-4bd0-82c1-da8e271b757f"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 195, 136])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bcf19bf5-acd4-44fd-9779-d38eb96f2b74, kind: Service { id: 34942533-fcec-49f1-bc91-3bae11367eac }, name: Name("nexus-34942533-fcec-49f1-bc91-3bae11367eac"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 8])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
38362025-06-04T01:33:28.414ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
38372025-06-04T01:33:28.462ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9416cc4b-b3a2-4400-aef1-a0187fb75ea1", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
38382025-06-04T01:33:28.462ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d switch = Switch0
38392025-06-04T01:33:28.462ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
38402025-06-04T01:33:28.462ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = f3041bd6-308e-4022-9f15-d44326b7d97d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "24d05ccc-1db4-46a5-914a-c08ae0a9b13e", "content-length": "682", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
38412025-06-04T01:33:28.462ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38422025-06-04T01:33:28.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38432025-06-04T01:33:28.466ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38442025-06-04T01:33:28.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38452025-06-04T01:33:28.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38462025-06-04T01:33:28.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38472025-06-04T01:33:28.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38482025-06-04T01:33:28.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38492025-06-04T01:33:28.472ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 4adf0198-0a8a-47c8-a668-dc22afd01e24, time_created: 2025-06-04T01:18:49.382391Z, time_modified: 2025-06-04T01:18:49.382391Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: f3041bd6-308e-4022-9f15-d44326b7d97d, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
38502025-06-04T01:33:28.472ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
38512025-06-04T01:33:28.479ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 747.621626ms iteration = 31 status = {}
38522025-06-04T01:33:28.502ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38532025-06-04T01:33:28.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38542025-06-04T01:33:28.504ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38552025-06-04T01:33:28.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38562025-06-04T01:33:28.506ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38572025-06-04T01:33:28.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 4adf0198-0a8a-47c8-a668-dc22afd01e24, lookup_type: ById(4adf0198-0a8a-47c8-a668-dc22afd01e24) } result = Ok(())
38582025-06-04T01:33:28.509ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
38592025-06-04T01:33:28.509ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 4adf0198-0a8a-47c8-a668-dc22afd01e24 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "aeb6687c-4453-4322-9ee1-194da6dde97a", "date": "Wed, 04 Jun 2025 01:33:28 GMT"} })
38602025-06-04T01:33:28.511ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:495
38612025-06-04T01:33:28.511ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 779.759938ms
38622025-06-04T01:33:28.511ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 779.769228ms iteration = 4 status = {}
38632025-06-04T01:33:28.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c14e1c-f0d2-4c85-8297-932033fbabbd roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38642025-06-04T01:33:28.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c14e1c-f0d2-4c85-8297-932033fbabbd resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38652025-06-04T01:33:28.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1820 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c3c14e1c-f0d2-4c85-8297-932033fbabbd response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38662025-06-04T01:33:29.676ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 606df1fb-e668-478e-8990-9104fd0d3ee4 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38672025-06-04T01:33:29.676ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 606df1fb-e668-478e-8990-9104fd0d3ee4 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38682025-06-04T01:33:29.679ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2780 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 606df1fb-e668-478e-8990-9104fd0d3ee4 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38692025-06-04T01:33:30.458ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 538 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = ee70449b-d718-40c1-9a51-25ff78f8ce3f response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
38702025-06-04T01:33:30.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 547e44fb-1a6f-4c97-af1e-b918c85cd233 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38712025-06-04T01:33:30.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 547e44fb-1a6f-4c97-af1e-b918c85cd233 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38722025-06-04T01:33:30.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2466 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 547e44fb-1a6f-4c97-af1e-b918c85cd233 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38732025-06-04T01:33:31.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 048bf344-3ff0-4ff0-8448-38fc5bcfc2f0 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38742025-06-04T01:33:31.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 048bf344-3ff0-4ff0-8448-38fc5bcfc2f0 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38752025-06-04T01:33:31.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 048bf344-3ff0-4ff0-8448-38fc5bcfc2f0 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38762025-06-04T01:33:32.725ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c27e4508-64be-44b6-8172-ff6f6ec36a63 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38772025-06-04T01:33:32.726ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c27e4508-64be-44b6-8172-ff6f6ec36a63 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38782025-06-04T01:33:32.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2648 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = c27e4508-64be-44b6-8172-ff6f6ec36a63 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38792025-06-04T01:33:33.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 86ffa481-9b70-4fcf-83d2-f0c79488e508 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38802025-06-04T01:33:33.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 86ffa481-9b70-4fcf-83d2-f0c79488e508 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38812025-06-04T01:33:33.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 86ffa481-9b70-4fcf-83d2-f0c79488e508 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38822025-06-04T01:33:34.756ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = abd223eb-6a80-4833-80b9-f347069932f6 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38832025-06-04T01:33:34.756ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = abd223eb-6a80-4833-80b9-f347069932f6 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38842025-06-04T01:33:34.758ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2671 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = abd223eb-6a80-4833-80b9-f347069932f6 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38852025-06-04T01:33:35.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3f3bfd91-a4aa-4174-8f80-51ff86f5745e roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38862025-06-04T01:33:35.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3f3bfd91-a4aa-4174-8f80-51ff86f5745e resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38872025-06-04T01:33:35.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2477 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3f3bfd91-a4aa-4174-8f80-51ff86f5745e response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38882025-06-04T01:33:36.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3d39db81-fd98-412a-8083-d2ff7462954c roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38892025-06-04T01:33:36.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3d39db81-fd98-412a-8083-d2ff7462954c resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38902025-06-04T01:33:36.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2289 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = 3d39db81-fd98-412a-8083-d2ff7462954c response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
38912025-06-04T01:33:37.730ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 92 reason = Timeout
38922025-06-04T01:33:37.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38932025-06-04T01:33:37.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38942025-06-04T01:33:37.731ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 92 reason = Timeout
38952025-06-04T01:33:37.731ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38962025-06-04T01:33:37.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38972025-06-04T01:33:37.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38982025-06-04T01:33:37.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38992025-06-04T01:33:37.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39002025-06-04T01:33:37.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39012025-06-04T01:33:37.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39022025-06-04T01:33:37.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39032025-06-04T01:33:37.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39042025-06-04T01:33:37.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 380e7fae-5d5a-479a-b890-8108906f1718, lookup_type: ById(380e7fae-5d5a-479a-b890-8108906f1718) } result = Ok(())
39052025-06-04T01:33:37.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39062025-06-04T01:33:37.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39072025-06-04T01:33:37.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 380e7fae-5d5a-479a-b890-8108906f1718 original_time_created = 2025-06-04 01:18:14.518554 UTC target_id = 380e7fae-5d5a-479a-b890-8108906f1718 time_created = 2025-06-04 01:18:14.518554 UTC
39082025-06-04T01:33:37.758ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.129819ms iteration = 92 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"380e7fae-5d5a-479a-b890-8108906f1718","time_created":"2025-06-04T01:18:14.518554Z"}
39092025-06-04T01:33:37.769ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39102025-06-04T01:33:37.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39112025-06-04T01:33:37.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.300941ms iteration = 92 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39122025-06-04T01:33:37.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b80ac07b-0c1d-4681-a9dc-3f9c9d286426 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39132025-06-04T01:33:37.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b80ac07b-0c1d-4681-a9dc-3f9c9d286426 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39142025-06-04T01:33:37.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1511 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = b80ac07b-0c1d-4681-a9dc-3f9c9d286426 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39152025-06-04T01:33:38.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bc83731f-035f-4925-ba3f-a5336b28fae8 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39162025-06-04T01:33:38.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bc83731f-035f-4925-ba3f-a5336b28fae8 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39172025-06-04T01:33:38.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = bc83731f-035f-4925-ba3f-a5336b28fae8 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39182025-06-04T01:33:39.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d266a2ae-c53c-41ba-b0cd-028064f699b9 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39192025-06-04T01:33:39.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d266a2ae-c53c-41ba-b0cd-028064f699b9 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39202025-06-04T01:33:39.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d266a2ae-c53c-41ba-b0cd-028064f699b9 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39212025-06-04T01:33:40.451ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:45454
39222025-06-04T01:33:40.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 12a16f6f-55eb-4d1f-ab62-e276eac93333 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
39232025-06-04T01:33:40.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 12a16f6f-55eb-4d1f-ab62-e276eac93333 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
39242025-06-04T01:33:40.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2511 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 12a16f6f-55eb-4d1f-ab62-e276eac93333 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?sort_by=id_ascending
39252025-06-04T01:33:40.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 3f099c26-8f58-4394-b930-38ce79284ac8 roles = RoleSet { roles: {} } uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
39262025-06-04T01:33:40.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 3f099c26-8f58-4394-b930-38ce79284ac8 resource = Database result = Ok(()) uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
39272025-06-04T01:33:40.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2450 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:45454 req_id = 3f099c26-8f58-4394-b930-38ce79284ac8 response_code = 200 uri = /metrics/collectors/b0f0765d-1e87-4904-b423-1a97eb2acce1/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMDIyOTU2My0yMDg2LTRiZDAtODJjMS1kYThlMjcxYjc1N2YifX0%3D
39282025-06-04T01:33:40.458ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 543 local_addr = [fd00:1122:3344:101::b]:48618 method = GET remote_addr = [fd00:1122:3344:101::d]:62867 req_id = 3d3dc36a-fd31-4534-b7f9-afdfd9c76db5 response_code = 200 uri = /7716fd65-e841-4c70-8dfc-958c8d9211a5
39292025-06-04T01:33:40.851ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ae6257-e9fb-4d23-a6ca-3b530e02b2a1 roles = RoleSet { roles: {} } uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39302025-06-04T01:33:40.851ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ae6257-e9fb-4d23-a6ca-3b530e02b2a1 resource = Database result = Ok(()) uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24
39312025-06-04T01:33:40.853ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1991 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:39659 req_id = d2ae6257-e9fb-4d23-a6ca-3b530e02b2a1 response_code = 200 uri = /probes/4adf0198-0a8a-47c8-a668-dc22afd01e24