Max level shown:
12025-04-16T04:00:03.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4cb95682-f058-4ee3-8177-0629a49592f4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
22025-04-16T04:00:03.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4cb95682-f058-4ee3-8177-0629a49592f4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
32025-04-16T04:00:03.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2085 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4cb95682-f058-4ee3-8177-0629a49592f4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
42025-04-16T04:00:04.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d241f4f-b874-4bc3-be6c-c4481bc941ef roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
52025-04-16T04:00:04.372ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d241f4f-b874-4bc3-be6c-c4481bc941ef resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62025-04-16T04:00:04.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2026 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d241f4f-b874-4bc3-be6c-c4481bc941ef response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72025-04-16T04:00:05.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e41669c6-150a-43d8-aa98-a8406e9b72b5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
82025-04-16T04:00:05.394ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e41669c6-150a-43d8-aa98-a8406e9b72b5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
92025-04-16T04:00:05.396ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1864 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e41669c6-150a-43d8-aa98-a8406e9b72b5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
102025-04-16T04:00:06.416ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a06d4bb-8504-4d75-8748-7805aceede85 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
112025-04-16T04:00:06.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a06d4bb-8504-4d75-8748-7805aceede85 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
122025-04-16T04:00:06.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a06d4bb-8504-4d75-8748-7805aceede85 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
132025-04-16T04:00:07.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1afd15-38e8-407e-be25-aa480ad4eb52 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
142025-04-16T04:00:07.441ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1afd15-38e8-407e-be25-aa480ad4eb52 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
152025-04-16T04:00:07.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2100 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1afd15-38e8-407e-be25-aa480ad4eb52 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
162025-04-16T04:00:08.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c21d734-c1f9-4ddb-85b0-c3aff3e296ea roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
172025-04-16T04:00:08.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c21d734-c1f9-4ddb-85b0-c3aff3e296ea resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
182025-04-16T04:00:08.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2336 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c21d734-c1f9-4ddb-85b0-c3aff3e296ea response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
192025-04-16T04:00:08.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 5a3d03ed-45b2-42c3-a1e2-b01c7ee439ac response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
202025-04-16T04:00:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
212025-04-16T04:00:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
222025-04-16T04:00:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232025-04-16T04:00:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
242025-04-16T04:00:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
252025-04-16T04:00:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
262025-04-16T04:00:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272025-04-16T04:00:09.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
282025-04-16T04:00:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
292025-04-16T04:00:09.451ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.466228ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
302025-04-16T04:00:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
312025-04-16T04:00:09.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
322025-04-16T04:00:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
332025-04-16T04:00:09.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
342025-04-16T04:00:09.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
352025-04-16T04:00:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
362025-04-16T04:00:09.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
372025-04-16T04:00:09.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
382025-04-16T04:00:09.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
392025-04-16T04:00:09.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
402025-04-16T04:00:09.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.320251ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
412025-04-16T04:00:09.485ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4103f73-54c8-474d-8075-2d89cfcc6f7b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
422025-04-16T04:00:09.485ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4103f73-54c8-474d-8075-2d89cfcc6f7b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
432025-04-16T04:00:09.487ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1732 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4103f73-54c8-474d-8075-2d89cfcc6f7b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
442025-04-16T04:00:10.505ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49712117-2de0-4bbf-9d75-bdb9841df065 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
452025-04-16T04:00:10.505ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49712117-2de0-4bbf-9d75-bdb9841df065 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
462025-04-16T04:00:10.507ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2514 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49712117-2de0-4bbf-9d75-bdb9841df065 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
472025-04-16T04:00:11.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69c40867-dfe3-4b8f-bf12-cf22c63734fb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
482025-04-16T04:00:11.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69c40867-dfe3-4b8f-bf12-cf22c63734fb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
492025-04-16T04:00:11.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1846 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69c40867-dfe3-4b8f-bf12-cf22c63734fb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
502025-04-16T04:00:12.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65ffa11b-2597-465f-a420-9d9dc59dc5b0 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
512025-04-16T04:00:12.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65ffa11b-2597-465f-a420-9d9dc59dc5b0 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
522025-04-16T04:00:12.552ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2264 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65ffa11b-2597-465f-a420-9d9dc59dc5b0 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
532025-04-16T04:00:13.573ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a22e6646-3376-4e68-ab81-270c339eb4b1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
542025-04-16T04:00:13.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a22e6646-3376-4e68-ab81-270c339eb4b1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
552025-04-16T04:00:13.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2487 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a22e6646-3376-4e68-ab81-270c339eb4b1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
562025-04-16T04:00:14.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e5c06a0-425b-4b52-a8ec-e491a5e5dff7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
572025-04-16T04:00:14.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e5c06a0-425b-4b52-a8ec-e491a5e5dff7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
582025-04-16T04:00:14.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1870 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e5c06a0-425b-4b52-a8ec-e491a5e5dff7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
592025-04-16T04:00:15.618ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33c26287-0e77-41c8-a49a-e3c1a197489f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
602025-04-16T04:00:15.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33c26287-0e77-41c8-a49a-e3c1a197489f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
612025-04-16T04:00:15.620ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1946 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33c26287-0e77-41c8-a49a-e3c1a197489f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
622025-04-16T04:00:16.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c17d7cc-a404-4662-b41b-e8efc841f6fa roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
632025-04-16T04:00:16.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c17d7cc-a404-4662-b41b-e8efc841f6fa resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
642025-04-16T04:00:16.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2118 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c17d7cc-a404-4662-b41b-e8efc841f6fa response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
652025-04-16T04:00:17.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f6d44139-5a4a-4f79-a543-326671ca3ff4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
662025-04-16T04:00:17.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f6d44139-5a4a-4f79-a543-326671ca3ff4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
672025-04-16T04:00:17.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2060 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f6d44139-5a4a-4f79-a543-326671ca3ff4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
682025-04-16T04:00:18.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 515 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 9b13d04b-45ec-43a4-9d80-1639fc449404 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
692025-04-16T04:00:18.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 35479d03-e4ce-437d-89cd-4956837d5e5f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
702025-04-16T04:00:18.684ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 35479d03-e4ce-437d-89cd-4956837d5e5f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
712025-04-16T04:00:18.686ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 35479d03-e4ce-437d-89cd-4956837d5e5f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
722025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
732025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
742025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
752025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
762025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
772025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
782025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
792025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
802025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
812025-04-16T04:00:19.447ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
822025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
832025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
842025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.01\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
852025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
862025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
872025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
882025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
892025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
902025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
912025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
922025-04-16T04:00:19.447ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
932025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
942025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
952025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
962025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
972025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
982025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
992025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1002025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
1012025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1022025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
1032025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
1042025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
1052025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1062025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
1072025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1082025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
1092025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1102025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
1112025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1122025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
1132025-04-16T04:00:19.447ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1142025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1152025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
1162025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1172025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
1182025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1192025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
1202025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1212025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
1222025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1232025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
1242025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1252025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
1262025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1272025-04-16T04:00:19.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1282025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
1292025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1302025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
1312025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1322025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1332025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
1342025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1352025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
1362025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
1372025-04-16T04:00:19.448ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:50:19.447936174 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1382025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1392025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
1402025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1412025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1422025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
1432025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1442025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1452025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1462025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1472025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1482025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1492025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1502025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1512025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1522025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1532025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1542025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1552025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1562025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1572025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1582025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1592025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1602025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1612025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1622025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1632025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1642025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1652025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1662025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1672025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1682025-04-16T04:00:19.448ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 2909, tv_nsec: 729297437 } })
1692025-04-16T04:00:19.448ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 2909, tv_nsec: 729353758 } })
1702025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1712025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1722025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1732025-04-16T04:00:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1742025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1752025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.141702ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1762025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1772025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.867267ms iteration = 7 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1782025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1792025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1802025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1812025-04-16T04:00:19.449ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1822025-04-16T04:00:19.450ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1832025-04-16T04:00:19.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1842025-04-16T04:00:19.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1852025-04-16T04:00:19.451ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2909, tv_nsec: 732486850 } })
1862025-04-16T04:00:19.451ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
1872025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1882025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1892025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1902025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1912025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1922025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.773144ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1932025-04-16T04:00:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1942025-04-16T04:00:19.454ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1952025-04-16T04:00:19.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2909, tv_nsec: 736251354 } })
1962025-04-16T04:00:19.455ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
1972025-04-16T04:00:19.456ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1982025-04-16T04:00:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1992025-04-16T04:00:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2002025-04-16T04:00:19.456ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2012025-04-16T04:00:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2022025-04-16T04:00:19.458ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.700871ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
2032025-04-16T04:00:19.458ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2042025-04-16T04:00:19.460ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2052025-04-16T04:00:19.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2062025-04-16T04:00:19.460ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2072025-04-16T04:00:19.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2082025-04-16T04:00:19.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2092025-04-16T04:00:19.462ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2102025-04-16T04:00:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2112025-04-16T04:00:19.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2122025-04-16T04:00:19.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2132025-04-16T04:00:19.465ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2142025-04-16T04:00:19.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2152025-04-16T04:00:19.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2162025-04-16T04:00:19.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2172025-04-16T04:00:19.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2182025-04-16T04:00:19.471ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192025-04-16T04:00:19.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2202025-04-16T04:00:19.473ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:50:19.447936174 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
2212025-04-16T04:00:19.473ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 25.524344ms iteration = 7 status = {"expiration":"2025-04-16T03:50:19.447936174Z","pruned":[]}
2222025-04-16T04:00:19.473ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2232025-04-16T04:00:19.473ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2242025-04-16T04:00:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2252025-04-16T04:00:19.474ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2262025-04-16T04:00:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2272025-04-16T04:00:19.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2282025-04-16T04:00:19.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2292025-04-16T04:00:19.476ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 29.095384ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
2302025-04-16T04:00:19.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2312025-04-16T04:00:19.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2322025-04-16T04:00:19.477ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2332025-04-16T04:00:19.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2342025-04-16T04:00:19.480ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2352025-04-16T04:00:19.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2362025-04-16T04:00:19.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2372025-04-16T04:00:19.482ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2382025-04-16T04:00:19.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2392025-04-16T04:00:19.482ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2402025-04-16T04:00:19.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2412025-04-16T04:00:19.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2422025-04-16T04:00:19.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2432025-04-16T04:00:19.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2442025-04-16T04:00:19.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 40.057581ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2452025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2462025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2472025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2482025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2492025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2502025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2512025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2522025-04-16T04:00:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2532025-04-16T04:00:19.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2542025-04-16T04:00:19.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
2552025-04-16T04:00:19.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2562025-04-16T04:00:19.493ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2572025-04-16T04:00:19.493ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2582025-04-16T04:00:19.493ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 46.227272ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2592025-04-16T04:00:19.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2602025-04-16T04:00:19.495ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2612025-04-16T04:00:19.496ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2909, tv_nsec: 777638111 } })
2622025-04-16T04:00:19.497ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
2632025-04-16T04:00:19.497ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2642025-04-16T04:00:19.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2652025-04-16T04:00:19.499ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2662025-04-16T04:00:19.500ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 53.120099ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
2672025-04-16T04:00:19.501ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2682025-04-16T04:00:19.501ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2692025-04-16T04:00:19.501ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2702025-04-16T04:00:19.501ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2712025-04-16T04:00:19.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2722025-04-16T04:00:19.503ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2732025-04-16T04:00:19.503ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2742025-04-16T04:00:19.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2752025-04-16T04:00:19.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2762025-04-16T04:00:19.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2772025-04-16T04:00:19.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 60.516804ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
2782025-04-16T04:00:19.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2792025-04-16T04:00:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2802025-04-16T04:00:19.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2812025-04-16T04:00:19.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2822025-04-16T04:00:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2832025-04-16T04:00:19.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2842025-04-16T04:00:19.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2852025-04-16T04:00:19.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2862025-04-16T04:00:19.512ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2872025-04-16T04:00:19.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2882025-04-16T04:00:19.513ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2892025-04-16T04:00:19.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2902025-04-16T04:00:19.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2912025-04-16T04:00:19.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2922025-04-16T04:00:19.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2932025-04-16T04:00:19.515ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2942025-04-16T04:00:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2952025-04-16T04:00:19.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2962025-04-16T04:00:19.519ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.96412ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2972025-04-16T04:00:19.520ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2982025-04-16T04:00:19.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2992025-04-16T04:00:19.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3002025-04-16T04:00:19.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3012025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3022025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
3032025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3042025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.537881ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
3052025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3062025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3072025-04-16T04:00:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3082025-04-16T04:00:19.525ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3092025-04-16T04:00:19.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3102025-04-16T04:00:19.526ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3112025-04-16T04:00:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3122025-04-16T04:00:19.530ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3132025-04-16T04:00:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3142025-04-16T04:00:19.532ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3152025-04-16T04:00:19.532ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3162025-04-16T04:00:19.532ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3172025-04-16T04:00:19.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3182025-04-16T04:00:19.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3192025-04-16T04:00:19.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3202025-04-16T04:00:19.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3212025-04-16T04:00:19.534ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3222025-04-16T04:00:19.534ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
3232025-04-16T04:00:19.534ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 87.241162ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3242025-04-16T04:00:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3252025-04-16T04:00:19.535ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3262025-04-16T04:00:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3272025-04-16T04:00:19.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
3282025-04-16T04:00:19.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3292025-04-16T04:00:19.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3302025-04-16T04:00:19.546ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3312025-04-16T04:00:19.546ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 99.235169ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3322025-04-16T04:00:19.547ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3332025-04-16T04:00:19.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3342025-04-16T04:00:19.549ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 101.592135ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3352025-04-16T04:00:19.553ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3362025-04-16T04:00:19.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3372025-04-16T04:00:19.558ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3382025-04-16T04:00:19.558ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3392025-04-16T04:00:19.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3402025-04-16T04:00:19.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3412025-04-16T04:00:19.560ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3422025-04-16T04:00:19.560ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3432025-04-16T04:00:19.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3442025-04-16T04:00:19.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3452025-04-16T04:00:19.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 124.870564ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3462025-04-16T04:00:19.592ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3472025-04-16T04:00:19.605ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
3482025-04-16T04:00:19.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3492025-04-16T04:00:19.605ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3502025-04-16T04:00:19.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3512025-04-16T04:00:19.608ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff571686-2a8a-44a1-bd2a-5b311b567e2a", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
3522025-04-16T04:00:19.608ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3532025-04-16T04:00:19.614ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3542025-04-16T04:00:19.664ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 217.452323ms iteration = 15 status = {"physical_disks_added":0}
3552025-04-16T04:00:19.673ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3562025-04-16T04:00:19.674ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3572025-04-16T04:00:19.674ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0bda202-fadf-4b6b-abaf-1c702458852a", "content-length": "372", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
3582025-04-16T04:00:19.674ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3592025-04-16T04:00:19.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3602025-04-16T04:00:19.677ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 229.372148ms iteration = 4 status = {"nsuccess":5}
3612025-04-16T04:00:19.678ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 230.550371ms iteration = 13 status = 0
3622025-04-16T04:00:19.680ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3632025-04-16T04:00:19.680ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3642025-04-16T04:00:19.681ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652025-04-16T04:00:19.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3662025-04-16T04:00:19.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3672025-04-16T04:00:19.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3682025-04-16T04:00:19.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3692025-04-16T04:00:19.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3702025-04-16T04:00:19.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3712025-04-16T04:00:19.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3722025-04-16T04:00:19.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3732025-04-16T04:00:19.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3742025-04-16T04:00:19.702ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
3752025-04-16T04:00:19.702ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 254.784759ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3762025-04-16T04:00:19.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3680a5d3-cc13-4a57-8f08-d61513a8a65d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
3772025-04-16T04:00:19.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3680a5d3-cc13-4a57-8f08-d61513a8a65d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
3782025-04-16T04:00:19.707ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
3792025-04-16T04:00:19.707ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3802025-04-16T04:00:19.707ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a6fd53a-dbf8-44fb-b36c-5baf03d1b416", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
3812025-04-16T04:00:19.707ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3822025-04-16T04:00:19.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2883 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3680a5d3-cc13-4a57-8f08-d61513a8a65d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
3832025-04-16T04:00:19.710ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
3842025-04-16T04:00:19.719ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3852025-04-16T04:00:19.723ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
3862025-04-16T04:00:19.723ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3872025-04-16T04:00:19.724ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0db84a86-e953-4067-8b79-61f367630d77", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
3882025-04-16T04:00:19.724ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3892025-04-16T04:00:19.728ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3902025-04-16T04:00:19.734ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3912025-04-16T04:00:19.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3922025-04-16T04:00:19.738ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3932025-04-16T04:00:19.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3942025-04-16T04:00:19.739ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3952025-04-16T04:00:19.739ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3962025-04-16T04:00:19.739ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 291.783139ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3972025-04-16T04:00:19.741ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
3982025-04-16T04:00:19.741ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3992025-04-16T04:00:19.741ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
4002025-04-16T04:00:19.741ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 293.898762ms iteration = 7 status = {"generation":1}
4012025-04-16T04:00:19.744ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
4022025-04-16T04:00:19.744ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 297.107885ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4032025-04-16T04:00:19.746ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 299.006312ms iteration = 13 status = {}
4042025-04-16T04:00:19.759ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
4052025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4062025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "331888cb-6ae4-4e7a-8d3b-88921264d360", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
4072025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4082025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4092025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
4102025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4112025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4122025-04-16T04:00:19.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4132025-04-16T04:00:19.761ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4142025-04-16T04:00:19.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4152025-04-16T04:00:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4162025-04-16T04:00:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4172025-04-16T04:00:19.764ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4182025-04-16T04:00:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4192025-04-16T04:00:19.765ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4202025-04-16T04:00:19.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4212025-04-16T04:00:19.766ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
4222025-04-16T04:00:19.766ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4232025-04-16T04:00:19.766ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
4242025-04-16T04:00:19.766ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 319.440086ms iteration = 7 status = {"generation":2}
4252025-04-16T04:00:19.768ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4262025-04-16T04:00:19.769ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4272025-04-16T04:00:19.769ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d831741d-03e7-454f-9b03-ba38e1d7355c", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
4282025-04-16T04:00:19.770ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 323.168009ms iteration = 13 status = {}
4292025-04-16T04:00:19.791ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4302025-04-16T04:00:19.792ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "06ae9255-bc6e-463a-8127-7922fe7dfff0", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4312025-04-16T04:00:19.792ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4322025-04-16T04:00:19.793ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "de443f2f-05b5-41d0-ac81-971ebc96543b", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4332025-04-16T04:00:19.793ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4342025-04-16T04:00:19.795ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
4352025-04-16T04:00:19.795ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 347.506039ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
4362025-04-16T04:00:19.830ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
4372025-04-16T04:00:19.830ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 382.603802ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4382025-04-16T04:00:19.841ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4392025-04-16T04:00:19.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4402025-04-16T04:00:19.846ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4412025-04-16T04:00:19.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4422025-04-16T04:00:19.851ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 404.393602ms iteration = 13 status = null
4432025-04-16T04:00:19.854ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4442025-04-16T04:00:19.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4452025-04-16T04:00:19.857ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4462025-04-16T04:00:19.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4472025-04-16T04:00:19.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4482025-04-16T04:00:19.863ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4492025-04-16T04:00:19.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4502025-04-16T04:00:19.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4512025-04-16T04:00:19.875ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4522025-04-16T04:00:19.935ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4532025-04-16T04:00:19.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4542025-04-16T04:00:19.938ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
4552025-04-16T04:00:19.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4562025-04-16T04:00:19.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4572025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4582025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
4592025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4602025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4612025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4622025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4632025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4642025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4652025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4662025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4672025-04-16T04:00:19.952ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
4682025-04-16T04:00:19.952ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4692025-04-16T04:00:19.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4702025-04-16T04:00:19.955ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
4712025-04-16T04:00:19.955ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4722025-04-16T04:00:19.973ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4732025-04-16T04:00:19.973ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "994d356b-ae47-41be-91b0-23b92c7ca5ea", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4742025-04-16T04:00:19.973ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4752025-04-16T04:00:19.974ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b39dd55d-d5db-483f-86d4-0ba277270c14", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4762025-04-16T04:00:19.974ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4772025-04-16T04:00:19.976ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "12b68f2a-78bc-4bf3-b450-c2faeb15a89b", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4782025-04-16T04:00:19.977ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4792025-04-16T04:00:19.979ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4802025-04-16T04:00:19.979ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 532.45141ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
4812025-04-16T04:00:20.025ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4822025-04-16T04:00:20.075ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6fc49858-9996-4f9d-ae39-aca3360cffa1", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
4832025-04-16T04:00:20.075ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
4842025-04-16T04:00:20.075ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4852025-04-16T04:00:20.076ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa2c9172-b574-431d-b4d4-ff6e4e8cbeff", "content-length": "682", "date": "Wed, 16 Apr 2025 04:00:19 GMT"} })
4862025-04-16T04:00:20.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4872025-04-16T04:00:20.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4882025-04-16T04:00:20.079ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4892025-04-16T04:00:20.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4902025-04-16T04:00:20.081ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6decb0c9-ae83-4e54-8cd2-7135787565dd", "content-length": "691", "date": "Wed, 16 Apr 2025 04:00:20 GMT"} })
4912025-04-16T04:00:20.081ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
4922025-04-16T04:00:20.081ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
4932025-04-16T04:00:20.081ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
4942025-04-16T04:00:20.083ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 635.586566ms iteration = 13 status = {}
4952025-04-16T04:00:20.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4962025-04-16T04:00:20.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4972025-04-16T04:00:20.086ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4982025-04-16T04:00:20.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4992025-04-16T04:00:20.093ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 646.097073ms iteration = 13 status = {}
5002025-04-16T04:00:20.497ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
5012025-04-16T04:00:20.497ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
5022025-04-16T04:00:20.527ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
5032025-04-16T04:00:20.527ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:58162
5042025-04-16T04:00:20.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:58162 req_id = f21ff0dc-415b-4a7d-83d7-27af70c0beea roles = RoleSet { roles: {} } uri = /metrics/producers
5052025-04-16T04:00:20.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:58162 req_id = f21ff0dc-415b-4a7d-83d7-27af70c0beea resource = Database result = Ok(()) uri = /metrics/producers
5062025-04-16T04:00:20.564ZINFOnexus: assigned collector to new producer collector_id = 0efaecbf-c4c4-456a-96b6-2cd10690be0f file = nexus/src/app/oximeter.rs:76 producer_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
5072025-04-16T04:00:20.564ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 36549 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:58162 req_id = f21ff0dc-415b-4a7d-83d7-27af70c0beea response_code = 201 uri = /metrics/producers
5082025-04-16T04:00:20.564ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "f21ff0dc-415b-4a7d-83d7-27af70c0beea", "content-length": "41", "date": "Wed, 16 Apr 2025 04:00:20 GMT"} })
5092025-04-16T04:00:20.565ZDEBGnexus (producer-registration-task): registered with nexus successfully
5102025-04-16T04:00:20.565ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
5112025-04-16T04:00:20.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b79a4d9f-0ff1-4c08-9598-0b5387345f41 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5122025-04-16T04:00:20.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b79a4d9f-0ff1-4c08-9598-0b5387345f41 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5132025-04-16T04:00:20.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2406 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b79a4d9f-0ff1-4c08-9598-0b5387345f41 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5142025-04-16T04:00:21.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10e3f650-f0f0-41f1-b205-453091e881a9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5152025-04-16T04:00:21.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10e3f650-f0f0-41f1-b205-453091e881a9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5162025-04-16T04:00:21.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10e3f650-f0f0-41f1-b205-453091e881a9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5172025-04-16T04:00:22.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17107671-2d99-421e-a7b1-abc87c42bd14 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5182025-04-16T04:00:22.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17107671-2d99-421e-a7b1-abc87c42bd14 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5192025-04-16T04:00:22.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17107671-2d99-421e-a7b1-abc87c42bd14 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5202025-04-16T04:00:23.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb1bd55f-eacd-4ee6-bd66-0266bca549ff roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5212025-04-16T04:00:23.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb1bd55f-eacd-4ee6-bd66-0266bca549ff resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5222025-04-16T04:00:23.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3156 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb1bd55f-eacd-4ee6-bd66-0266bca549ff response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5232025-04-16T04:00:24.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a3f473d-e194-4326-a3c6-637a1d156cf2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5242025-04-16T04:00:24.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a3f473d-e194-4326-a3c6-637a1d156cf2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5252025-04-16T04:00:24.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2419 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a3f473d-e194-4326-a3c6-637a1d156cf2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5262025-04-16T04:00:25.848ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cacf1de5-6347-43ad-81f7-a1adde63277a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5272025-04-16T04:00:25.848ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cacf1de5-6347-43ad-81f7-a1adde63277a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5282025-04-16T04:00:25.850ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2000 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cacf1de5-6347-43ad-81f7-a1adde63277a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5292025-04-16T04:00:26.869ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cae2021-cecb-4ffe-b7ea-a4d9276974d3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5302025-04-16T04:00:26.869ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cae2021-cecb-4ffe-b7ea-a4d9276974d3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5312025-04-16T04:00:26.871ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cae2021-cecb-4ffe-b7ea-a4d9276974d3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5322025-04-16T04:00:27.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4132964-421e-49b5-93cc-200b43b3160b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5332025-04-16T04:00:27.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4132964-421e-49b5-93cc-200b43b3160b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5342025-04-16T04:00:27.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2013 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4132964-421e-49b5-93cc-200b43b3160b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5352025-04-16T04:00:28.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 0dddbab2-7500-4ace-9a52-9976b0abd851 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
5362025-04-16T04:00:28.558ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 0dddbab2-7500-4ace-9a52-9976b0abd851 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
5372025-04-16T04:00:28.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2949 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 0dddbab2-7500-4ace-9a52-9976b0abd851 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
5382025-04-16T04:00:28.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 96775bcd-b238-410f-8f7f-878a05ac42f0 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
5392025-04-16T04:00:28.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 96775bcd-b238-410f-8f7f-878a05ac42f0 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
5402025-04-16T04:00:28.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1929 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 96775bcd-b238-410f-8f7f-878a05ac42f0 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
5412025-04-16T04:00:28.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 464 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = b524e33b-5733-4bc3-a6ba-63f0752332a9 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
5422025-04-16T04:00:28.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c0a2e2e-1b2d-41d5-8ce9-c13bc3025cc4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5432025-04-16T04:00:28.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c0a2e2e-1b2d-41d5-8ce9-c13bc3025cc4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5442025-04-16T04:00:28.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1783 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c0a2e2e-1b2d-41d5-8ce9-c13bc3025cc4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5452025-04-16T04:00:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
5462025-04-16T04:00:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
5472025-04-16T04:00:29.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5482025-04-16T04:00:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5492025-04-16T04:00:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5502025-04-16T04:00:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5512025-04-16T04:00:29.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5522025-04-16T04:00:29.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5532025-04-16T04:00:29.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5542025-04-16T04:00:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5552025-04-16T04:00:29.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5562025-04-16T04:00:29.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5572025-04-16T04:00:29.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5582025-04-16T04:00:29.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
5592025-04-16T04:00:29.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5602025-04-16T04:00:29.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5612025-04-16T04:00:29.470ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
5622025-04-16T04:00:29.470ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.903072ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
5632025-04-16T04:00:29.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5642025-04-16T04:00:29.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5652025-04-16T04:00:29.515ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.103834ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5662025-04-16T04:00:29.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 75da3f4d-8972-4fea-8ba1-599674f55146 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5672025-04-16T04:00:29.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 75da3f4d-8972-4fea-8ba1-599674f55146 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5682025-04-16T04:00:29.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1706 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 75da3f4d-8972-4fea-8ba1-599674f55146 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5692025-04-16T04:00:30.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8224c143-2839-42b9-a6d6-4b7a7d3aec56 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5702025-04-16T04:00:30.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8224c143-2839-42b9-a6d6-4b7a7d3aec56 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5712025-04-16T04:00:30.962ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8224c143-2839-42b9-a6d6-4b7a7d3aec56 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5722025-04-16T04:00:31.982ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1d15eb7c-bc25-42e9-a6ed-8d42a1f38b7c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5732025-04-16T04:00:31.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1d15eb7c-bc25-42e9-a6ed-8d42a1f38b7c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5742025-04-16T04:00:31.984ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1d15eb7c-bc25-42e9-a6ed-8d42a1f38b7c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5752025-04-16T04:00:33.003ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd3c18fc-aa52-4a92-938d-0dfb33b0ccf1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5762025-04-16T04:00:33.004ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd3c18fc-aa52-4a92-938d-0dfb33b0ccf1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5772025-04-16T04:00:33.006ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2758 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd3c18fc-aa52-4a92-938d-0dfb33b0ccf1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5782025-04-16T04:00:34.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 78cc35a5-8ecc-408d-ad88-4a9c991ae7aa roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5792025-04-16T04:00:34.027ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 78cc35a5-8ecc-408d-ad88-4a9c991ae7aa resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5802025-04-16T04:00:34.028ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1925 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 78cc35a5-8ecc-408d-ad88-4a9c991ae7aa response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5812025-04-16T04:00:35.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b0521fc3-c28f-4198-95fe-cf0e05bab9f6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5822025-04-16T04:00:35.049ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b0521fc3-c28f-4198-95fe-cf0e05bab9f6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5832025-04-16T04:00:35.050ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1756 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b0521fc3-c28f-4198-95fe-cf0e05bab9f6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5842025-04-16T04:00:36.070ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0315b8d1-5672-4f31-8ef1-69fc4558c15e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5852025-04-16T04:00:36.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0315b8d1-5672-4f31-8ef1-69fc4558c15e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5862025-04-16T04:00:36.072ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2067 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0315b8d1-5672-4f31-8ef1-69fc4558c15e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5872025-04-16T04:00:37.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 68d8af65-9277-48e1-ab7d-91b08f63a8cc roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5882025-04-16T04:00:37.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 68d8af65-9277-48e1-ab7d-91b08f63a8cc resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5892025-04-16T04:00:37.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2311 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 68d8af65-9277-48e1-ab7d-91b08f63a8cc response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5902025-04-16T04:00:38.114ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fce44fe6-bf96-48ee-9883-c7fe0f4a1c4e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5912025-04-16T04:00:38.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fce44fe6-bf96-48ee-9883-c7fe0f4a1c4e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5922025-04-16T04:00:38.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fce44fe6-bf96-48ee-9883-c7fe0f4a1c4e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5932025-04-16T04:00:38.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 513 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = fe09a834-8e91-4908-8a57-042b77e3c7ad response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
5942025-04-16T04:00:39.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fbe6a961-e7e6-49db-8eed-4c0285ba98a3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5952025-04-16T04:00:39.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fbe6a961-e7e6-49db-8eed-4c0285ba98a3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5962025-04-16T04:00:39.138ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2263 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fbe6a961-e7e6-49db-8eed-4c0285ba98a3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
5972025-04-16T04:00:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
5982025-04-16T04:00:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5992025-04-16T04:00:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
6002025-04-16T04:00:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6012025-04-16T04:00:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6022025-04-16T04:00:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6032025-04-16T04:00:39.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6042025-04-16T04:00:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6052025-04-16T04:00:39.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6062025-04-16T04:00:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6072025-04-16T04:00:39.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6082025-04-16T04:00:39.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6092025-04-16T04:00:39.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102025-04-16T04:00:39.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
6112025-04-16T04:00:39.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6122025-04-16T04:00:39.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6132025-04-16T04:00:39.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
6142025-04-16T04:00:39.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.851041ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
6152025-04-16T04:00:39.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6162025-04-16T04:00:39.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6172025-04-16T04:00:39.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.684476ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6182025-04-16T04:00:40.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88269c7e-ce41-4541-a98c-3df21a45d0f6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6192025-04-16T04:00:40.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88269c7e-ce41-4541-a98c-3df21a45d0f6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6202025-04-16T04:00:40.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1933 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88269c7e-ce41-4541-a98c-3df21a45d0f6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6212025-04-16T04:00:41.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3f869a0-1ad1-41ee-8bce-cd42cc747285 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6222025-04-16T04:00:41.181ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3f869a0-1ad1-41ee-8bce-cd42cc747285 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6232025-04-16T04:00:41.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2429 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3f869a0-1ad1-41ee-8bce-cd42cc747285 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6242025-04-16T04:00:42.204ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5045e428-c5c1-4ca1-b904-a947838cec9c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6252025-04-16T04:00:42.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5045e428-c5c1-4ca1-b904-a947838cec9c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6262025-04-16T04:00:42.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2572 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5045e428-c5c1-4ca1-b904-a947838cec9c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6272025-04-16T04:00:43.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b812f134-3176-4259-bb40-78dd867b4edd roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6282025-04-16T04:00:43.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b812f134-3176-4259-bb40-78dd867b4edd resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6292025-04-16T04:00:43.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2270 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b812f134-3176-4259-bb40-78dd867b4edd response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6302025-04-16T04:00:44.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4278d5df-f9bb-4ccc-acaa-7fdf9a00706c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6312025-04-16T04:00:44.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4278d5df-f9bb-4ccc-acaa-7fdf9a00706c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6322025-04-16T04:00:44.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4278d5df-f9bb-4ccc-acaa-7fdf9a00706c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6332025-04-16T04:00:45.268ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 50b41248-8ffb-4732-a57a-468b20ec340e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6342025-04-16T04:00:45.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 50b41248-8ffb-4732-a57a-468b20ec340e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6352025-04-16T04:00:45.271ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2921 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 50b41248-8ffb-4732-a57a-468b20ec340e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6362025-04-16T04:00:46.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7244a4b4-65d9-41d6-a23a-7dd7cd03cd6f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6372025-04-16T04:00:46.293ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7244a4b4-65d9-41d6-a23a-7dd7cd03cd6f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6382025-04-16T04:00:46.295ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1941 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7244a4b4-65d9-41d6-a23a-7dd7cd03cd6f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6392025-04-16T04:00:47.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 889469c8-bdca-4ac8-ae6d-829c838f0e00 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6402025-04-16T04:00:47.314ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 889469c8-bdca-4ac8-ae6d-829c838f0e00 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6412025-04-16T04:00:47.316ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2357 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 889469c8-bdca-4ac8-ae6d-829c838f0e00 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6422025-04-16T04:00:48.337ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6d6be740-ead9-4b5f-9b3e-dc83ef2d7567 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6432025-04-16T04:00:48.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6d6be740-ead9-4b5f-9b3e-dc83ef2d7567 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6442025-04-16T04:00:48.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2832 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6d6be740-ead9-4b5f-9b3e-dc83ef2d7567 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6452025-04-16T04:00:48.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 489 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = f9a7764b-0cf6-417e-af7c-e27c86821bba response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
6462025-04-16T04:00:49.359ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d0a9e1-d98e-48b7-b560-4dfca6f3c142 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6472025-04-16T04:00:49.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d0a9e1-d98e-48b7-b560-4dfca6f3c142 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6482025-04-16T04:00:49.361ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1679 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d0a9e1-d98e-48b7-b560-4dfca6f3c142 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
6492025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
6502025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
6512025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6522025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
6532025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6542025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6552025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
6562025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6572025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
6582025-04-16T04:00:49.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6592025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
6602025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6612025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
6622025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6632025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6642025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
6652025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
6662025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
6672025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6682025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6692025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
6702025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6712025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
6722025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6732025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
6742025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6752025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
6762025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6772025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
6782025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6792025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
6802025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
6812025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
6822025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6832025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
6842025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6852025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
6862025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
6872025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6882025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6892025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6902025-04-16T04:00:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6912025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6922025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6932025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6942025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6952025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6962025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6972025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6982025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6992025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7002025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7012025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7022025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7032025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7042025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
7052025-04-16T04:00:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7062025-04-16T04:00:49.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7072025-04-16T04:00:49.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7082025-04-16T04:00:49.449ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7092025-04-16T04:00:49.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102025-04-16T04:00:49.450ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.082531ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7112025-04-16T04:00:49.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7122025-04-16T04:00:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7132025-04-16T04:00:49.452ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7142025-04-16T04:00:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7152025-04-16T04:00:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7162025-04-16T04:00:49.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7172025-04-16T04:00:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7182025-04-16T04:00:49.454ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7192025-04-16T04:00:49.454ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7202025-04-16T04:00:49.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7212025-04-16T04:00:49.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2939, tv_nsec: 736462811 } })
7222025-04-16T04:00:49.455ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7232025-04-16T04:00:49.455ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
7242025-04-16T04:00:49.456ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2939, tv_nsec: 737750017 } })
7252025-04-16T04:00:49.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
7262025-04-16T04:00:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7272025-04-16T04:00:49.457ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7282025-04-16T04:00:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7292025-04-16T04:00:49.458ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7302025-04-16T04:00:49.458ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
7312025-04-16T04:00:49.458ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7322025-04-16T04:00:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7332025-04-16T04:00:49.458ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7342025-04-16T04:00:49.460ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7352025-04-16T04:00:49.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7362025-04-16T04:00:49.460ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7372025-04-16T04:00:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7382025-04-16T04:00:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7392025-04-16T04:00:49.463ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7402025-04-16T04:00:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7412025-04-16T04:00:49.466ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2939, tv_nsec: 747472438 } })
7422025-04-16T04:00:49.466ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
7432025-04-16T04:00:49.467ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7442025-04-16T04:00:49.467ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7452025-04-16T04:00:49.467ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7462025-04-16T04:00:49.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7472025-04-16T04:00:49.470ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7482025-04-16T04:00:49.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7492025-04-16T04:00:49.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7502025-04-16T04:00:49.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7512025-04-16T04:00:49.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7522025-04-16T04:00:49.493ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7532025-04-16T04:00:49.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7542025-04-16T04:00:49.493ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7552025-04-16T04:00:49.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7562025-04-16T04:00:49.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7572025-04-16T04:00:49.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 50.564208ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7582025-04-16T04:00:49.496ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7592025-04-16T04:00:49.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7602025-04-16T04:00:49.498ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.14816ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
7612025-04-16T04:00:49.500ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.805012ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
7622025-04-16T04:00:49.500ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7632025-04-16T04:00:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7642025-04-16T04:00:49.502ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7652025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.843402ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7662025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7672025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7682025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7692025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7702025-04-16T04:00:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7712025-04-16T04:00:49.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7722025-04-16T04:00:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7732025-04-16T04:00:49.505ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
7742025-04-16T04:00:49.505ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 58.975534ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7752025-04-16T04:00:49.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7762025-04-16T04:00:49.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7772025-04-16T04:00:49.507ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 61.059725ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
7782025-04-16T04:00:49.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7792025-04-16T04:00:49.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7802025-04-16T04:00:49.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7812025-04-16T04:00:49.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7822025-04-16T04:00:49.509ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7832025-04-16T04:00:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7842025-04-16T04:00:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
7852025-04-16T04:00:49.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7862025-04-16T04:00:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7872025-04-16T04:00:49.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
7882025-04-16T04:00:49.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7892025-04-16T04:00:49.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7902025-04-16T04:00:49.514ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 67.764138ms iteration = 14 status = 0
7912025-04-16T04:00:49.515ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7922025-04-16T04:00:49.515ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 69.263587ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7932025-04-16T04:00:49.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942025-04-16T04:00:49.521ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7952025-04-16T04:00:49.521ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7962025-04-16T04:00:49.521ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 75.278586ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7972025-04-16T04:00:49.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7982025-04-16T04:00:49.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7992025-04-16T04:00:49.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8002025-04-16T04:00:49.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
8012025-04-16T04:00:49.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.418758ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
8022025-04-16T04:00:49.530ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
8032025-04-16T04:00:49.531ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8042025-04-16T04:00:49.531ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0afd1675-4c94-4144-bc6e-bb4f20788f83", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:48 GMT"} })
8052025-04-16T04:00:49.531ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8062025-04-16T04:00:49.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8072025-04-16T04:00:49.532ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8082025-04-16T04:00:49.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8092025-04-16T04:00:49.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8102025-04-16T04:00:49.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8112025-04-16T04:00:49.538ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 91.769432ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8122025-04-16T04:00:49.552ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
8132025-04-16T04:00:49.552ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8142025-04-16T04:00:49.553ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4a49cdb-f344-4802-af8b-40ff0bccaaf8", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:48 GMT"} })
8152025-04-16T04:00:49.553ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8162025-04-16T04:00:49.553ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 107.499752ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
8172025-04-16T04:00:49.555ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8182025-04-16T04:00:49.583ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8192025-04-16T04:00:49.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8202025-04-16T04:00:49.587ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
8212025-04-16T04:00:49.587ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8222025-04-16T04:00:49.588ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2cba834-f5a2-4a5b-be83-8b8090ad3e18", "content-length": "26", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8232025-04-16T04:00:49.588ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8242025-04-16T04:00:49.590ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 143.961191ms iteration = 16 status = {"physical_disks_added":0}
8252025-04-16T04:00:49.590ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
8262025-04-16T04:00:49.649ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8272025-04-16T04:00:49.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8282025-04-16T04:00:49.652ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8292025-04-16T04:00:49.652ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 206.22136ms iteration = 14 status = {}
8302025-04-16T04:00:49.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8312025-04-16T04:00:49.654ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8322025-04-16T04:00:49.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8332025-04-16T04:00:49.655ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8342025-04-16T04:00:49.657ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8352025-04-16T04:00:49.657ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8362025-04-16T04:00:49.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8372025-04-16T04:00:49.685ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8382025-04-16T04:00:49.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8392025-04-16T04:00:49.688ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 242.116469ms iteration = 14 status = null
8402025-04-16T04:00:49.718ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8412025-04-16T04:00:49.718ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8422025-04-16T04:00:49.719ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "62812768-b7c2-43ce-b326-ef5714a4d02c", "content-length": "372", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8432025-04-16T04:00:49.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8442025-04-16T04:00:49.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8452025-04-16T04:00:49.745ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8462025-04-16T04:00:49.746ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f724a693-e1f3-439d-9d4c-435938a9e6d9", "content-length": "691", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8472025-04-16T04:00:49.746ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8482025-04-16T04:00:49.746ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8492025-04-16T04:00:49.746ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8502025-04-16T04:00:49.747ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 301.179145ms iteration = 14 status = {}
8512025-04-16T04:00:49.765ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8522025-04-16T04:00:49.765ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8532025-04-16T04:00:49.765ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "51703bed-5d59-4a4b-abe1-b3cf2fe0bd39", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8542025-04-16T04:00:49.765ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8552025-04-16T04:00:49.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8562025-04-16T04:00:49.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8572025-04-16T04:00:49.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8582025-04-16T04:00:49.771ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8592025-04-16T04:00:49.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8602025-04-16T04:00:49.774ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
8612025-04-16T04:00:49.774ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8622025-04-16T04:00:49.775ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ce21f516-fe28-4671-99b8-b761b140d006", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8632025-04-16T04:00:49.777ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 331.113926ms iteration = 14 status = {}
8642025-04-16T04:00:49.809ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8652025-04-16T04:00:49.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8662025-04-16T04:00:49.814ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
8672025-04-16T04:00:49.814ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8682025-04-16T04:00:49.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8692025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8702025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
8712025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8722025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8732025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8742025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8752025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8762025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8772025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8782025-04-16T04:00:49.827ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8792025-04-16T04:00:49.828ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
8802025-04-16T04:00:49.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8812025-04-16T04:00:49.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8822025-04-16T04:00:49.833ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
8832025-04-16T04:00:49.833ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8842025-04-16T04:00:49.966ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "391ab7e9-056b-47f8-9bc9-1f6cf5cccf02", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8852025-04-16T04:00:49.966ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
8862025-04-16T04:00:49.966ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8872025-04-16T04:00:49.966ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebead552-cfcc-43d4-b254-81a7e62ddd2b", "content-length": "682", "date": "Wed, 16 Apr 2025 04:00:49 GMT"} })
8882025-04-16T04:00:49.967ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8892025-04-16T04:00:49.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8902025-04-16T04:00:49.971ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8912025-04-16T04:00:49.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8922025-04-16T04:00:49.974ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8932025-04-16T04:00:49.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8942025-04-16T04:00:49.977ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8952025-04-16T04:00:49.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8962025-04-16T04:00:49.984ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 538.136512ms iteration = 14 status = {}
8972025-04-16T04:00:50.381ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8aa55684-2293-4226-b219-992254b9cd85 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
8982025-04-16T04:00:50.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8aa55684-2293-4226-b219-992254b9cd85 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
8992025-04-16T04:00:50.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2087 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8aa55684-2293-4226-b219-992254b9cd85 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9002025-04-16T04:00:51.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d51df846-5bba-4929-96b3-6ce60d9b11ee roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9012025-04-16T04:00:51.402ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d51df846-5bba-4929-96b3-6ce60d9b11ee resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9022025-04-16T04:00:51.404ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2320 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d51df846-5bba-4929-96b3-6ce60d9b11ee response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9032025-04-16T04:00:52.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76f67417-b3db-45f5-86ce-b7e4a1587cc6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9042025-04-16T04:00:52.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76f67417-b3db-45f5-86ce-b7e4a1587cc6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9052025-04-16T04:00:52.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2356 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76f67417-b3db-45f5-86ce-b7e4a1587cc6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9062025-04-16T04:00:53.446ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b8e63da3-f2bf-4b88-9bc4-73b0e96039d2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9072025-04-16T04:00:53.447ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b8e63da3-f2bf-4b88-9bc4-73b0e96039d2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9082025-04-16T04:00:53.449ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b8e63da3-f2bf-4b88-9bc4-73b0e96039d2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9092025-04-16T04:00:54.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16c56a9e-eb08-479d-b6fd-327be97d6596 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9102025-04-16T04:00:54.469ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16c56a9e-eb08-479d-b6fd-327be97d6596 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9112025-04-16T04:00:54.471ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2550 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16c56a9e-eb08-479d-b6fd-327be97d6596 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9122025-04-16T04:00:55.490ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e67d890-0d66-4919-bf3a-c37092e5d08f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9132025-04-16T04:00:55.491ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e67d890-0d66-4919-bf3a-c37092e5d08f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9142025-04-16T04:00:55.493ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2391 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1e67d890-0d66-4919-bf3a-c37092e5d08f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9152025-04-16T04:00:56.512ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 67993e54-4c46-4b15-95a9-f509a3827b61 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9162025-04-16T04:00:56.512ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 67993e54-4c46-4b15-95a9-f509a3827b61 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9172025-04-16T04:00:56.514ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2260 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 67993e54-4c46-4b15-95a9-f509a3827b61 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9182025-04-16T04:00:57.535ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = efece311-059d-4d05-a887-f7a3c16c38d5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9192025-04-16T04:00:57.535ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = efece311-059d-4d05-a887-f7a3c16c38d5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9202025-04-16T04:00:57.537ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1868 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = efece311-059d-4d05-a887-f7a3c16c38d5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9212025-04-16T04:00:58.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cba0d479-6236-44a6-b006-a8037b58e290 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9222025-04-16T04:00:58.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cba0d479-6236-44a6-b006-a8037b58e290 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9232025-04-16T04:00:58.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 33d1723b-6f22-420c-8e75-daab32bf28cf roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
9242025-04-16T04:00:58.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 33d1723b-6f22-420c-8e75-daab32bf28cf resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
9252025-04-16T04:00:58.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2280 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cba0d479-6236-44a6-b006-a8037b58e290 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9262025-04-16T04:00:58.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = 33d1723b-6f22-420c-8e75-daab32bf28cf response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
9272025-04-16T04:00:58.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = c7be4a8d-411b-4c90-b3aa-3a8c47eba7ef roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
9282025-04-16T04:00:58.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = c7be4a8d-411b-4c90-b3aa-3a8c47eba7ef resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
9292025-04-16T04:00:58.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2523 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62965 req_id = c7be4a8d-411b-4c90-b3aa-3a8c47eba7ef response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
9302025-04-16T04:00:58.565ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 761 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 9593dd28-1773-4c22-a45c-58b963c40882 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
9312025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
9322025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
9332025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9342025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9352025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9362025-04-16T04:00:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9372025-04-16T04:00:59.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9382025-04-16T04:00:59.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9392025-04-16T04:00:59.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402025-04-16T04:00:59.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.760205ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9412025-04-16T04:00:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9422025-04-16T04:00:59.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9432025-04-16T04:00:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9442025-04-16T04:00:59.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9452025-04-16T04:00:59.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9462025-04-16T04:00:59.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9472025-04-16T04:00:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
9482025-04-16T04:00:59.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9492025-04-16T04:00:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9502025-04-16T04:00:59.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
9512025-04-16T04:00:59.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.637887ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
9522025-04-16T04:00:59.578ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c161dd27-1fde-4ee1-ab23-356fb37d7826 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9532025-04-16T04:00:59.579ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c161dd27-1fde-4ee1-ab23-356fb37d7826 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9542025-04-16T04:00:59.580ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1789 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c161dd27-1fde-4ee1-ab23-356fb37d7826 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9552025-04-16T04:01:00.602ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 674af0c6-eafc-4677-9d00-995d2a1a4f1d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9562025-04-16T04:01:00.603ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 674af0c6-eafc-4677-9d00-995d2a1a4f1d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9572025-04-16T04:01:00.605ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2326 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 674af0c6-eafc-4677-9d00-995d2a1a4f1d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9582025-04-16T04:01:01.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84e01657-7e4e-4d5d-85b4-f633f43b3e04 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9592025-04-16T04:01:01.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84e01657-7e4e-4d5d-85b4-f633f43b3e04 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9602025-04-16T04:01:01.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1943 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84e01657-7e4e-4d5d-85b4-f633f43b3e04 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9612025-04-16T04:01:02.645ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 282c07dd-87f2-4e5e-bb93-9aec88097591 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9622025-04-16T04:01:02.645ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 282c07dd-87f2-4e5e-bb93-9aec88097591 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9632025-04-16T04:01:02.647ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2338 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 282c07dd-87f2-4e5e-bb93-9aec88097591 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9642025-04-16T04:01:03.667ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbffae59-38e0-45d2-9c10-335591c063e6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9652025-04-16T04:01:03.667ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbffae59-38e0-45d2-9c10-335591c063e6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9662025-04-16T04:01:03.669ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2347 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbffae59-38e0-45d2-9c10-335591c063e6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9672025-04-16T04:01:04.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98025285-f93b-439c-a955-8d51adc7289a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9682025-04-16T04:01:04.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98025285-f93b-439c-a955-8d51adc7289a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9692025-04-16T04:01:04.692ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98025285-f93b-439c-a955-8d51adc7289a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9702025-04-16T04:01:05.712ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c893aa40-c451-4c7e-bd6c-2fa598084882 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9712025-04-16T04:01:05.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c893aa40-c451-4c7e-bd6c-2fa598084882 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9722025-04-16T04:01:05.714ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c893aa40-c451-4c7e-bd6c-2fa598084882 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9732025-04-16T04:01:06.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d5a0a13-8260-4c0e-902a-7c8de7aa81cf roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9742025-04-16T04:01:06.735ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d5a0a13-8260-4c0e-902a-7c8de7aa81cf resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9752025-04-16T04:01:06.736ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1639 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d5a0a13-8260-4c0e-902a-7c8de7aa81cf response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9762025-04-16T04:01:07.756ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17d90a47-d62f-4cce-a053-e9c0ab3643a9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9772025-04-16T04:01:07.757ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17d90a47-d62f-4cce-a053-e9c0ab3643a9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9782025-04-16T04:01:07.759ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 17d90a47-d62f-4cce-a053-e9c0ab3643a9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9792025-04-16T04:01:08.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 481 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 7329769f-4718-4470-8fdd-35e233264245 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
9802025-04-16T04:01:08.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 965bb922-e5d0-4037-8ae2-06d56b020fa6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9812025-04-16T04:01:08.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 965bb922-e5d0-4037-8ae2-06d56b020fa6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9822025-04-16T04:01:08.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2014 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 965bb922-e5d0-4037-8ae2-06d56b020fa6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
9832025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
9842025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
9852025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9862025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9872025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9882025-04-16T04:01:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9892025-04-16T04:01:09.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9902025-04-16T04:01:09.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9912025-04-16T04:01:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9922025-04-16T04:01:09.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.280095ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9932025-04-16T04:01:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9942025-04-16T04:01:09.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9952025-04-16T04:01:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9962025-04-16T04:01:09.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9972025-04-16T04:01:09.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9982025-04-16T04:01:09.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992025-04-16T04:01:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
10002025-04-16T04:01:09.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10012025-04-16T04:01:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10022025-04-16T04:01:09.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
10032025-04-16T04:01:09.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.006104ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
10042025-04-16T04:01:09.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4d0a3b3c-1630-4be8-951e-af4ee12e924a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10052025-04-16T04:01:09.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4d0a3b3c-1630-4be8-951e-af4ee12e924a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10062025-04-16T04:01:09.803ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2231 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4d0a3b3c-1630-4be8-951e-af4ee12e924a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10072025-04-16T04:01:10.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61fce7eb-1aa2-47a9-8d64-b91bd7d35d2f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10082025-04-16T04:01:10.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61fce7eb-1aa2-47a9-8d64-b91bd7d35d2f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10092025-04-16T04:01:10.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1645 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61fce7eb-1aa2-47a9-8d64-b91bd7d35d2f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10102025-04-16T04:01:11.845ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a32d3950-f8cd-404d-a3d3-bcd22f56670a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10112025-04-16T04:01:11.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a32d3950-f8cd-404d-a3d3-bcd22f56670a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10122025-04-16T04:01:11.847ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a32d3950-f8cd-404d-a3d3-bcd22f56670a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10132025-04-16T04:01:12.867ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9b9051ad-711f-440a-9d70-454e01803742 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10142025-04-16T04:01:12.867ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9b9051ad-711f-440a-9d70-454e01803742 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10152025-04-16T04:01:12.869ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9b9051ad-711f-440a-9d70-454e01803742 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10162025-04-16T04:01:13.890ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28f3405f-2e67-4643-8c51-834da6a03940 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10172025-04-16T04:01:13.890ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28f3405f-2e67-4643-8c51-834da6a03940 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10182025-04-16T04:01:13.892ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28f3405f-2e67-4643-8c51-834da6a03940 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10192025-04-16T04:01:14.912ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d9cf3448-c1fc-4d56-a664-18b293f3c464 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10202025-04-16T04:01:14.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d9cf3448-c1fc-4d56-a664-18b293f3c464 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10212025-04-16T04:01:14.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2293 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d9cf3448-c1fc-4d56-a664-18b293f3c464 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10222025-04-16T04:01:15.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 55b585d6-d9ce-4697-bbd9-8a08d75b9d08 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10232025-04-16T04:01:15.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 55b585d6-d9ce-4697-bbd9-8a08d75b9d08 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10242025-04-16T04:01:15.936ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1974 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 55b585d6-d9ce-4697-bbd9-8a08d75b9d08 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10252025-04-16T04:01:16.960ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2cda9fb-d5f8-4970-8c54-958df005b736 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10262025-04-16T04:01:16.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2cda9fb-d5f8-4970-8c54-958df005b736 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10272025-04-16T04:01:16.962ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2324 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2cda9fb-d5f8-4970-8c54-958df005b736 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10282025-04-16T04:01:17.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4f80c1f-f6bc-4507-add6-13c7b2cac914 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10292025-04-16T04:01:17.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4f80c1f-f6bc-4507-add6-13c7b2cac914 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10302025-04-16T04:01:17.982ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2010 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e4f80c1f-f6bc-4507-add6-13c7b2cac914 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10312025-04-16T04:01:18.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 535 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 6184a9a9-7beb-440c-909e-450adf7989b4 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
10322025-04-16T04:01:19.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4e74f66-7503-4877-97c6-a75c560f5852 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10332025-04-16T04:01:19.001ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4e74f66-7503-4877-97c6-a75c560f5852 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10342025-04-16T04:01:19.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2258 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4e74f66-7503-4877-97c6-a75c560f5852 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
10352025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
10362025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
10372025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10382025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
10392025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
10402025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
10412025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10422025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
10432025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10442025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
10452025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
10462025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10472025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
10482025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
10492025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10502025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10512025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
10522025-04-16T04:01:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
10532025-04-16T04:01:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10542025-04-16T04:01:19.447ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
10552025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.75\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
10562025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
10572025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10582025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
10592025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10602025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10612025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10622025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
10632025-04-16T04:01:19.447ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
10642025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
10652025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10662025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10672025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
10682025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10692025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
10702025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10712025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
10722025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10732025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
10742025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10752025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10762025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
10772025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10782025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
10792025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10802025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
10812025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10822025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
10832025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10842025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10852025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10862025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
10872025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
10882025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10892025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10902025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
10912025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
10922025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10932025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
10942025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10952025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
10962025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
10972025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
10982025-04-16T04:01:19.447ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:51:19.447164870 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
10992025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
11002025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11012025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
11022025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11032025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11042025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
11052025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11062025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11072025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11082025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11092025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11102025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
11112025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11122025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
11132025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11142025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11152025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11162025-04-16T04:01:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 2969, tv_nsec: 728724961 } })
11172025-04-16T04:01:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 2969, tv_nsec: 728731731 } })
11182025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
11192025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11202025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11212025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11222025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11232025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
11242025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
11252025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
11262025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11272025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11282025-04-16T04:01:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11292025-04-16T04:01:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
11302025-04-16T04:01:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 941.898\u{b5}s iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
11312025-04-16T04:01:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
11322025-04-16T04:01:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.413227ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
11332025-04-16T04:01:19.449ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2969, tv_nsec: 731034457 } })
11342025-04-16T04:01:19.450ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
11352025-04-16T04:01:19.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11362025-04-16T04:01:19.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11372025-04-16T04:01:19.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.164812ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
11382025-04-16T04:01:19.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11392025-04-16T04:01:19.452ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.263024ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
11402025-04-16T04:01:19.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11412025-04-16T04:01:19.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11422025-04-16T04:01:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11432025-04-16T04:01:19.456ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11442025-04-16T04:01:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11452025-04-16T04:01:19.459ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462025-04-16T04:01:19.460ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11472025-04-16T04:01:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11482025-04-16T04:01:19.462ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11492025-04-16T04:01:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11502025-04-16T04:01:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11512025-04-16T04:01:19.462ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11522025-04-16T04:01:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11532025-04-16T04:01:19.464ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.012445ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
11542025-04-16T04:01:19.465ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2969, tv_nsec: 746980162 } })
11552025-04-16T04:01:19.465ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11562025-04-16T04:01:19.466ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
11572025-04-16T04:01:19.466ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:51:19.447164870 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
11582025-04-16T04:01:19.466ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 19.123098ms iteration = 8 status = {"expiration":"2025-04-16T03:51:19.447164870Z","pruned":[]}
11592025-04-16T04:01:19.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11602025-04-16T04:01:19.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11612025-04-16T04:01:19.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11622025-04-16T04:01:19.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11632025-04-16T04:01:19.470ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11642025-04-16T04:01:19.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11652025-04-16T04:01:19.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11662025-04-16T04:01:19.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11672025-04-16T04:01:19.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11682025-04-16T04:01:19.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11692025-04-16T04:01:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11702025-04-16T04:01:19.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11712025-04-16T04:01:19.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11722025-04-16T04:01:19.503ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11732025-04-16T04:01:19.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742025-04-16T04:01:19.506ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11752025-04-16T04:01:19.506ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11762025-04-16T04:01:19.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11772025-04-16T04:01:19.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11782025-04-16T04:01:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
11792025-04-16T04:01:19.507ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
11802025-04-16T04:01:19.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11812025-04-16T04:01:19.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11822025-04-16T04:01:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11832025-04-16T04:01:19.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11842025-04-16T04:01:19.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11852025-04-16T04:01:19.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11862025-04-16T04:01:19.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11872025-04-16T04:01:19.509ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11882025-04-16T04:01:19.509ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11892025-04-16T04:01:19.509ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 62.84947ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11902025-04-16T04:01:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11912025-04-16T04:01:19.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11922025-04-16T04:01:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11932025-04-16T04:01:19.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942025-04-16T04:01:19.511ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11952025-04-16T04:01:19.512ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11962025-04-16T04:01:19.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11972025-04-16T04:01:19.513ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 66.88305ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11982025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11992025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12002025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12012025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12022025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12032025-04-16T04:01:19.514ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.722247ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
12042025-04-16T04:01:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12052025-04-16T04:01:19.515ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12062025-04-16T04:01:19.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12072025-04-16T04:01:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12082025-04-16T04:01:19.515ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 69.016682ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
12092025-04-16T04:01:19.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12102025-04-16T04:01:19.516ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12112025-04-16T04:01:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12122025-04-16T04:01:19.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12132025-04-16T04:01:19.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12142025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12152025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12162025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12172025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12182025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12192025-04-16T04:01:19.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12202025-04-16T04:01:19.519ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12212025-04-16T04:01:19.519ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12222025-04-16T04:01:19.519ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12232025-04-16T04:01:19.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12242025-04-16T04:01:19.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12252025-04-16T04:01:19.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 73.277547ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12262025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12272025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12282025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12292025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12302025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12312025-04-16T04:01:19.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12322025-04-16T04:01:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12332025-04-16T04:01:19.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2969, tv_nsec: 805205641 } })
12342025-04-16T04:01:19.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
12352025-04-16T04:01:19.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12362025-04-16T04:01:19.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12372025-04-16T04:01:19.524ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12382025-04-16T04:01:19.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12392025-04-16T04:01:19.527ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 80.724573ms iteration = 17 status = {"physical_disks_added":0}
12402025-04-16T04:01:19.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12412025-04-16T04:01:19.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12422025-04-16T04:01:19.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12432025-04-16T04:01:19.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12442025-04-16T04:01:19.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12452025-04-16T04:01:19.530ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12462025-04-16T04:01:19.530ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12472025-04-16T04:01:19.530ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12482025-04-16T04:01:19.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12492025-04-16T04:01:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12502025-04-16T04:01:19.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
12512025-04-16T04:01:19.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12522025-04-16T04:01:19.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12532025-04-16T04:01:19.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12542025-04-16T04:01:19.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12552025-04-16T04:01:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12562025-04-16T04:01:19.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12572025-04-16T04:01:19.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12582025-04-16T04:01:19.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12592025-04-16T04:01:19.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12602025-04-16T04:01:19.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12612025-04-16T04:01:19.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12622025-04-16T04:01:19.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12632025-04-16T04:01:19.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12642025-04-16T04:01:19.555ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12652025-04-16T04:01:19.556ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12662025-04-16T04:01:19.556ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
12672025-04-16T04:01:19.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12682025-04-16T04:01:19.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12692025-04-16T04:01:19.556ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
12702025-04-16T04:01:19.556ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 110.024782ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
12712025-04-16T04:01:19.557ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
12722025-04-16T04:01:19.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
12732025-04-16T04:01:19.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
12742025-04-16T04:01:19.558ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12752025-04-16T04:01:19.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12762025-04-16T04:01:19.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 111.821507ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12772025-04-16T04:01:19.559ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
12782025-04-16T04:01:19.559ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12792025-04-16T04:01:19.560ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "88a4cea0-342b-4208-8480-10aebab814f3", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:18 GMT"} })
12802025-04-16T04:01:19.560ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12812025-04-16T04:01:19.560ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
12822025-04-16T04:01:19.560ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
12832025-04-16T04:01:19.560ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 113.44869ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12842025-04-16T04:01:19.561ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 114.588072ms iteration = 15 status = 0
12852025-04-16T04:01:19.562ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12862025-04-16T04:01:19.576ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12872025-04-16T04:01:19.577ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12882025-04-16T04:01:19.577ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 130.924154ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12892025-04-16T04:01:19.581ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
12902025-04-16T04:01:19.585ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 137.904202ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12912025-04-16T04:01:19.598ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
12922025-04-16T04:01:19.598ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 151.576282ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12932025-04-16T04:01:19.600ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12942025-04-16T04:01:19.600ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
12952025-04-16T04:01:19.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12962025-04-16T04:01:19.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12972025-04-16T04:01:19.604ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12982025-04-16T04:01:19.604ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12992025-04-16T04:01:19.604ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 157.738653ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
13002025-04-16T04:01:19.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13012025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13022025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13032025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13042025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13052025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13062025-04-16T04:01:19.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13072025-04-16T04:01:19.610ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13082025-04-16T04:01:19.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13092025-04-16T04:01:19.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13102025-04-16T04:01:19.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13112025-04-16T04:01:19.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122025-04-16T04:01:19.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13132025-04-16T04:01:19.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13142025-04-16T04:01:19.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13152025-04-16T04:01:19.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13162025-04-16T04:01:19.618ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13172025-04-16T04:01:19.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13182025-04-16T04:01:19.621ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
13192025-04-16T04:01:19.621ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13202025-04-16T04:01:19.622ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1ededbf-d47c-4d42-8a94-41bd014d2428", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13212025-04-16T04:01:19.622ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13222025-04-16T04:01:19.624ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13232025-04-16T04:01:19.673ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13242025-04-16T04:01:19.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13252025-04-16T04:01:19.725ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
13262025-04-16T04:01:19.725ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 278.206382ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
13272025-04-16T04:01:19.730ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
13282025-04-16T04:01:19.730ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13292025-04-16T04:01:19.731ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f01cf943-1487-4712-9cce-0e5dcabeda57", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13302025-04-16T04:01:19.731ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13312025-04-16T04:01:19.732ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
13322025-04-16T04:01:19.736ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13332025-04-16T04:01:19.742ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13342025-04-16T04:01:19.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13352025-04-16T04:01:19.759ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13362025-04-16T04:01:19.759ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b7a7fc33-1907-4314-8ea6-3bfab48b797d", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
13372025-04-16T04:01:19.759ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
13382025-04-16T04:01:19.759ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13392025-04-16T04:01:19.759ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13402025-04-16T04:01:19.760ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "514cc837-520c-42fe-b3e1-420226221fe1", "content-length": "372", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13412025-04-16T04:01:19.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13422025-04-16T04:01:19.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13432025-04-16T04:01:19.761ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "be014d7c-5533-4b18-bdb4-e9d13da70ae5", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
13442025-04-16T04:01:19.761ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
13452025-04-16T04:01:19.762ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
13462025-04-16T04:01:19.762ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13472025-04-16T04:01:19.762ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
13482025-04-16T04:01:19.762ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 315.323214ms iteration = 8 status = {"generation":2}
13492025-04-16T04:01:19.762ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13502025-04-16T04:01:19.763ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
13512025-04-16T04:01:19.763ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 316.12946ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
13522025-04-16T04:01:19.763ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 316.813453ms iteration = 15 status = null
13532025-04-16T04:01:19.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13542025-04-16T04:01:19.764ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13552025-04-16T04:01:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13562025-04-16T04:01:19.768ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
13572025-04-16T04:01:19.768ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13582025-04-16T04:01:19.768ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
13592025-04-16T04:01:19.768ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 321.143909ms iteration = 8 status = {"generation":1}
13602025-04-16T04:01:19.771ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13612025-04-16T04:01:19.774ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13622025-04-16T04:01:19.774ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13632025-04-16T04:01:19.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13642025-04-16T04:01:19.776ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
13652025-04-16T04:01:19.776ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13662025-04-16T04:01:19.777ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "cfb95f53-f9b2-44f9-a041-f113e085eab4", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13672025-04-16T04:01:19.777ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13682025-04-16T04:01:19.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13692025-04-16T04:01:19.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13702025-04-16T04:01:19.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13712025-04-16T04:01:19.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13722025-04-16T04:01:19.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13732025-04-16T04:01:19.785ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 338.631444ms iteration = 15 status = {}
13742025-04-16T04:01:19.786ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13752025-04-16T04:01:19.786ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13762025-04-16T04:01:19.787ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a5c9c4a6-20f1-4378-8ad2-d52e93baf8c0", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13772025-04-16T04:01:19.789ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 342.499261ms iteration = 15 status = {}
13782025-04-16T04:01:19.807ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
13792025-04-16T04:01:19.807ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 360.091407ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
13802025-04-16T04:01:19.894ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13812025-04-16T04:01:19.895ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "44ae4c58-917b-44e4-b15d-67a914babbaf", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13822025-04-16T04:01:19.895ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13832025-04-16T04:01:19.897ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ad3a4978-f172-44a4-842d-6c2c946c9b28", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13842025-04-16T04:01:19.897ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13852025-04-16T04:01:19.899ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a2eb2909-602f-4097-b424-00dd61d4a726", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13862025-04-16T04:01:19.899ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13872025-04-16T04:01:19.900ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13882025-04-16T04:01:19.900ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 454.262166ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13892025-04-16T04:01:19.925ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13902025-04-16T04:01:19.925ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "73cc1bfd-14f6-4ba0-97df-2938344a7639", "content-length": "691", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
13912025-04-16T04:01:19.925ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
13922025-04-16T04:01:19.925ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
13932025-04-16T04:01:19.925ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
13942025-04-16T04:01:19.927ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 480.523805ms iteration = 15 status = {}
13952025-04-16T04:01:19.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13962025-04-16T04:01:19.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13972025-04-16T04:01:19.960ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
13982025-04-16T04:01:19.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13992025-04-16T04:01:19.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14002025-04-16T04:01:19.969ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
14012025-04-16T04:01:19.969ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
14022025-04-16T04:01:19.969ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
14032025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
14042025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14052025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14062025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14072025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14082025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14092025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14102025-04-16T04:01:19.970ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
14112025-04-16T04:01:19.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14122025-04-16T04:01:19.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14132025-04-16T04:01:19.973ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
14142025-04-16T04:01:19.973ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14152025-04-16T04:01:20.077ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b12d5a4e-26f3-4f0a-bd85-66d396661bd0", "date": "Wed, 16 Apr 2025 04:01:19 GMT"} })
14162025-04-16T04:01:20.078ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
14172025-04-16T04:01:20.078ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14182025-04-16T04:01:20.078ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 896a23a7-5e24-474d-aa9a-ab10c076d201 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14192025-04-16T04:01:20.078ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 896a23a7-5e24-474d-aa9a-ab10c076d201 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14202025-04-16T04:01:20.078ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f413e110-7448-476f-9692-469d7091ba7a", "content-length": "682", "date": "Wed, 16 Apr 2025 04:01:20 GMT"} })
14212025-04-16T04:01:20.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14222025-04-16T04:01:20.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14232025-04-16T04:01:20.080ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2379 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 896a23a7-5e24-474d-aa9a-ab10c076d201 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14242025-04-16T04:01:20.082ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14252025-04-16T04:01:20.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14262025-04-16T04:01:20.086ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14272025-04-16T04:01:20.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14282025-04-16T04:01:20.089ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14292025-04-16T04:01:20.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14302025-04-16T04:01:20.097ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 650.422448ms iteration = 15 status = {}
14312025-04-16T04:01:21.195ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a3143d3-7d96-43bf-a64f-aa3fc1d5e9b1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14322025-04-16T04:01:21.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a3143d3-7d96-43bf-a64f-aa3fc1d5e9b1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14332025-04-16T04:01:21.198ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2470 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a3143d3-7d96-43bf-a64f-aa3fc1d5e9b1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14342025-04-16T04:01:22.218ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f84aa39-21c0-4dfa-b94a-87c96f3ef49e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14352025-04-16T04:01:22.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f84aa39-21c0-4dfa-b94a-87c96f3ef49e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14362025-04-16T04:01:22.220ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2798 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f84aa39-21c0-4dfa-b94a-87c96f3ef49e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14372025-04-16T04:01:23.241ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 023f4f0c-87d3-4a27-8861-4c04fda29d49 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14382025-04-16T04:01:23.241ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 023f4f0c-87d3-4a27-8861-4c04fda29d49 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14392025-04-16T04:01:23.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2058 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 023f4f0c-87d3-4a27-8861-4c04fda29d49 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14402025-04-16T04:01:24.263ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e08ea3c2-3854-43fe-a345-0a3f8a162c1a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14412025-04-16T04:01:24.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e08ea3c2-3854-43fe-a345-0a3f8a162c1a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14422025-04-16T04:01:24.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2077 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e08ea3c2-3854-43fe-a345-0a3f8a162c1a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14432025-04-16T04:01:25.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c29baa64-c233-46db-a87c-a56dcc28514c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14442025-04-16T04:01:25.286ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c29baa64-c233-46db-a87c-a56dcc28514c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14452025-04-16T04:01:25.288ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2549 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c29baa64-c233-46db-a87c-a56dcc28514c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14462025-04-16T04:01:26.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1985b02a-17f3-47cc-af62-c89f391755e9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14472025-04-16T04:01:26.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1985b02a-17f3-47cc-af62-c89f391755e9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14482025-04-16T04:01:26.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1985b02a-17f3-47cc-af62-c89f391755e9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14492025-04-16T04:01:27.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77b6396e-8fa6-4022-a07e-08e8a12c7661 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14502025-04-16T04:01:27.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77b6396e-8fa6-4022-a07e-08e8a12c7661 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14512025-04-16T04:01:27.333ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2294 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77b6396e-8fa6-4022-a07e-08e8a12c7661 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14522025-04-16T04:01:28.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 886e0494-6abf-4eef-bb69-14d6527cf71b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14532025-04-16T04:01:28.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 886e0494-6abf-4eef-bb69-14d6527cf71b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14542025-04-16T04:01:28.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2022 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 886e0494-6abf-4eef-bb69-14d6527cf71b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14552025-04-16T04:01:28.565ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 452 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 8cf6eeeb-222a-4de0-b69c-a75311ae31f5 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
14562025-04-16T04:01:29.376ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e98c9901-3387-4063-a2b1-73ab7011476e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14572025-04-16T04:01:29.376ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e98c9901-3387-4063-a2b1-73ab7011476e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14582025-04-16T04:01:29.378ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e98c9901-3387-4063-a2b1-73ab7011476e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14592025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
14602025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
14612025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14622025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14632025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14642025-04-16T04:01:29.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14652025-04-16T04:01:29.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14662025-04-16T04:01:29.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672025-04-16T04:01:29.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14682025-04-16T04:01:29.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.090531ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14692025-04-16T04:01:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14702025-04-16T04:01:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14712025-04-16T04:01:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14722025-04-16T04:01:29.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14732025-04-16T04:01:29.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14742025-04-16T04:01:29.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14752025-04-16T04:01:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
14762025-04-16T04:01:29.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14772025-04-16T04:01:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14782025-04-16T04:01:29.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
14792025-04-16T04:01:29.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.015195ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
14802025-04-16T04:01:30.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = da17e397-0475-4609-908e-ad67343ff9ef roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14812025-04-16T04:01:30.398ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = da17e397-0475-4609-908e-ad67343ff9ef resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14822025-04-16T04:01:30.399ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = da17e397-0475-4609-908e-ad67343ff9ef response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14832025-04-16T04:01:31.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30879c01-2b6a-4aaa-a74c-fc7d55596cde roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14842025-04-16T04:01:31.422ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30879c01-2b6a-4aaa-a74c-fc7d55596cde resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14852025-04-16T04:01:31.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2200 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30879c01-2b6a-4aaa-a74c-fc7d55596cde response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14862025-04-16T04:01:32.443ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 087246c7-356c-4fe3-a197-a9e674ff3341 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14872025-04-16T04:01:32.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 087246c7-356c-4fe3-a197-a9e674ff3341 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14882025-04-16T04:01:32.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2328 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 087246c7-356c-4fe3-a197-a9e674ff3341 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14892025-04-16T04:01:33.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfb7ee93-7bab-43ff-a316-fd02f2982d33 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14902025-04-16T04:01:33.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfb7ee93-7bab-43ff-a316-fd02f2982d33 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14912025-04-16T04:01:33.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1718 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfb7ee93-7bab-43ff-a316-fd02f2982d33 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14922025-04-16T04:01:34.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2958cd1e-21f4-4480-b3cf-33207378e836 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14932025-04-16T04:01:34.486ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2958cd1e-21f4-4480-b3cf-33207378e836 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14942025-04-16T04:01:34.488ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2958cd1e-21f4-4480-b3cf-33207378e836 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14952025-04-16T04:01:35.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28eb2e19-efe7-4987-97c9-e5a9f77b7a1f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14962025-04-16T04:01:35.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28eb2e19-efe7-4987-97c9-e5a9f77b7a1f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14972025-04-16T04:01:35.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2278 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28eb2e19-efe7-4987-97c9-e5a9f77b7a1f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14982025-04-16T04:01:36.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70482f91-dd3d-469e-8713-35cce6219120 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
14992025-04-16T04:01:36.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70482f91-dd3d-469e-8713-35cce6219120 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15002025-04-16T04:01:36.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2582 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70482f91-dd3d-469e-8713-35cce6219120 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15012025-04-16T04:01:37.551ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8be9dcb0-945a-420a-995f-6bbae985bd62 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15022025-04-16T04:01:37.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8be9dcb0-945a-420a-995f-6bbae985bd62 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15032025-04-16T04:01:37.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2529 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8be9dcb0-945a-420a-995f-6bbae985bd62 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15042025-04-16T04:01:38.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 549 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 9d1bca86-a319-42f4-b033-c8e559bcd63d response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
15052025-04-16T04:01:38.573ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 292cf688-3229-441d-9c9a-6d0f73a3f987 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15062025-04-16T04:01:38.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 292cf688-3229-441d-9c9a-6d0f73a3f987 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15072025-04-16T04:01:38.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1725 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 292cf688-3229-441d-9c9a-6d0f73a3f987 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15082025-04-16T04:01:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
15092025-04-16T04:01:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
15102025-04-16T04:01:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15112025-04-16T04:01:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15122025-04-16T04:01:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15132025-04-16T04:01:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15142025-04-16T04:01:39.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152025-04-16T04:01:39.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15162025-04-16T04:01:39.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15172025-04-16T04:01:39.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.168182ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15182025-04-16T04:01:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15192025-04-16T04:01:39.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15202025-04-16T04:01:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15212025-04-16T04:01:39.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15222025-04-16T04:01:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15232025-04-16T04:01:39.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15242025-04-16T04:01:39.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
15252025-04-16T04:01:39.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15262025-04-16T04:01:39.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15272025-04-16T04:01:39.464ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
15282025-04-16T04:01:39.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.404644ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
15292025-04-16T04:01:39.596ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff06774e-9801-422d-80a4-711eebb11c36 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15302025-04-16T04:01:39.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff06774e-9801-422d-80a4-711eebb11c36 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15312025-04-16T04:01:39.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff06774e-9801-422d-80a4-711eebb11c36 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15322025-04-16T04:01:40.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3312ad4d-8213-4fb2-8716-98c4273629d6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15332025-04-16T04:01:40.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3312ad4d-8213-4fb2-8716-98c4273629d6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15342025-04-16T04:01:40.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2691 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3312ad4d-8213-4fb2-8716-98c4273629d6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15352025-04-16T04:01:41.641ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66a73a45-4cd8-4815-8cea-0922a7fc8de6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15362025-04-16T04:01:41.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66a73a45-4cd8-4815-8cea-0922a7fc8de6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15372025-04-16T04:01:41.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2513 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66a73a45-4cd8-4815-8cea-0922a7fc8de6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15382025-04-16T04:01:42.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66432a6f-1947-4beb-a2ee-e12ce01f6fe7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15392025-04-16T04:01:42.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66432a6f-1947-4beb-a2ee-e12ce01f6fe7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15402025-04-16T04:01:42.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 66432a6f-1947-4beb-a2ee-e12ce01f6fe7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15412025-04-16T04:01:43.557ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:65386
15422025-04-16T04:01:43.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = 9d2f8951-18df-4997-8751-d08a1d4977e4 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
15432025-04-16T04:01:43.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = 9d2f8951-18df-4997-8751-d08a1d4977e4 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
15442025-04-16T04:01:43.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2832 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = 9d2f8951-18df-4997-8751-d08a1d4977e4 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
15452025-04-16T04:01:43.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = a6bc15f5-ed56-4c53-ac23-166cee5fcdd7 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
15462025-04-16T04:01:43.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = a6bc15f5-ed56-4c53-ac23-166cee5fcdd7 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
15472025-04-16T04:01:43.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1699 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65386 req_id = a6bc15f5-ed56-4c53-ac23-166cee5fcdd7 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
15482025-04-16T04:01:43.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f339a4c-3523-4144-9cf3-0b785966aa8d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15492025-04-16T04:01:43.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f339a4c-3523-4144-9cf3-0b785966aa8d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15502025-04-16T04:01:43.695ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f339a4c-3523-4144-9cf3-0b785966aa8d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15512025-04-16T04:01:44.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a21569a-e6bd-4b6a-9150-0bbdb10b7dae roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15522025-04-16T04:01:44.715ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a21569a-e6bd-4b6a-9150-0bbdb10b7dae resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15532025-04-16T04:01:44.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a21569a-e6bd-4b6a-9150-0bbdb10b7dae response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15542025-04-16T04:01:45.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5f6cf82-a1e4-4c38-a19f-8c66ae0c61ab roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15552025-04-16T04:01:45.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5f6cf82-a1e4-4c38-a19f-8c66ae0c61ab resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15562025-04-16T04:01:45.741ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1970 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5f6cf82-a1e4-4c38-a19f-8c66ae0c61ab response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15572025-04-16T04:01:46.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5459ff43-3ac2-458a-a142-3dad19058f17 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15582025-04-16T04:01:46.761ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5459ff43-3ac2-458a-a142-3dad19058f17 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15592025-04-16T04:01:46.763ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2207 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5459ff43-3ac2-458a-a142-3dad19058f17 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15602025-04-16T04:01:47.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e38cff47-a1fc-4f5b-9c25-e7effb3d5da1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15612025-04-16T04:01:47.783ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e38cff47-a1fc-4f5b-9c25-e7effb3d5da1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15622025-04-16T04:01:47.785ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2226 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e38cff47-a1fc-4f5b-9c25-e7effb3d5da1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15632025-04-16T04:01:48.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 518 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 35a2e4b8-b1d5-4924-8068-bf79c8c6584b response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
15642025-04-16T04:01:48.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef6aba01-c88d-40f4-938b-161743baea23 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15652025-04-16T04:01:48.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef6aba01-c88d-40f4-938b-161743baea23 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15662025-04-16T04:01:48.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef6aba01-c88d-40f4-938b-161743baea23 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
15672025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
15682025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
15692025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15702025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15712025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
15722025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
15732025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15742025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15752025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
15762025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15772025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15782025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
15792025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15802025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
15812025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15822025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
15832025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
15842025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
15852025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15862025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
15872025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15882025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15892025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
15902025-04-16T04:01:49.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15912025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
15922025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15932025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15942025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
15952025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
15962025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15972025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15982025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
15992025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
16002025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
16012025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16022025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
16032025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16042025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
16052025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
16062025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16072025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16082025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16092025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
16102025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16112025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16122025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16132025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16142025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16152025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16162025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16172025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16182025-04-16T04:01:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
16192025-04-16T04:01:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16202025-04-16T04:01:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
16212025-04-16T04:01:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16222025-04-16T04:01:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16232025-04-16T04:01:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16242025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16252025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.00762ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
16262025-04-16T04:01:49.449ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16272025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
16282025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16292025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.098871ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
16302025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16312025-04-16T04:01:49.449ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322025-04-16T04:01:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16332025-04-16T04:01:49.451ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16342025-04-16T04:01:49.451ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2999, tv_nsec: 732712862 } })
16352025-04-16T04:01:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16362025-04-16T04:01:49.451ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
16372025-04-16T04:01:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16382025-04-16T04:01:49.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16392025-04-16T04:01:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16402025-04-16T04:01:49.454ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16412025-04-16T04:01:49.454ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16422025-04-16T04:01:49.455ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16432025-04-16T04:01:49.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2999, tv_nsec: 736525747 } })
16442025-04-16T04:01:49.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16452025-04-16T04:01:49.455ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
16462025-04-16T04:01:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16472025-04-16T04:01:49.457ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16482025-04-16T04:01:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16492025-04-16T04:01:49.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502025-04-16T04:01:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16512025-04-16T04:01:49.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16522025-04-16T04:01:49.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16532025-04-16T04:01:49.460ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16542025-04-16T04:01:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16552025-04-16T04:01:49.461ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16562025-04-16T04:01:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16572025-04-16T04:01:49.461ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16582025-04-16T04:01:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16592025-04-16T04:01:49.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16602025-04-16T04:01:49.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16612025-04-16T04:01:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16622025-04-16T04:01:49.463ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16632025-04-16T04:01:49.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16642025-04-16T04:01:49.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16652025-04-16T04:01:49.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16662025-04-16T04:01:49.466ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2999, tv_nsec: 747780740 } })
16672025-04-16T04:01:49.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16682025-04-16T04:01:49.467ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
16692025-04-16T04:01:49.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16702025-04-16T04:01:49.467ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 20.832631ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16712025-04-16T04:01:49.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16722025-04-16T04:01:49.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
16732025-04-16T04:01:49.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16742025-04-16T04:01:49.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16752025-04-16T04:01:49.474ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16762025-04-16T04:01:49.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16772025-04-16T04:01:49.485ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16782025-04-16T04:01:49.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16792025-04-16T04:01:49.486ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16802025-04-16T04:01:49.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16812025-04-16T04:01:49.487ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822025-04-16T04:01:49.488ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16832025-04-16T04:01:49.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.917047ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16842025-04-16T04:01:49.488ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
16852025-04-16T04:01:49.488ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.310955ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
16862025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16872025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16882025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16892025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16902025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16912025-04-16T04:01:49.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16922025-04-16T04:01:49.491ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16932025-04-16T04:01:49.494ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16942025-04-16T04:01:49.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16952025-04-16T04:01:49.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16962025-04-16T04:01:49.497ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16972025-04-16T04:01:49.498ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 51.719581ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16982025-04-16T04:01:49.500ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16992025-04-16T04:01:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17002025-04-16T04:01:49.502ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 55.955275ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
17012025-04-16T04:01:49.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17022025-04-16T04:01:49.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17032025-04-16T04:01:49.505ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.23775ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
17042025-04-16T04:01:49.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17052025-04-16T04:01:49.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17062025-04-16T04:01:49.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17072025-04-16T04:01:49.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17082025-04-16T04:01:49.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17092025-04-16T04:01:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17102025-04-16T04:01:49.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17112025-04-16T04:01:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17122025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17132025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17142025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17152025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17162025-04-16T04:01:49.510ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17172025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
17182025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17192025-04-16T04:01:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17202025-04-16T04:01:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17212025-04-16T04:01:49.511ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17222025-04-16T04:01:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17232025-04-16T04:01:49.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17242025-04-16T04:01:49.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
17252025-04-16T04:01:49.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17262025-04-16T04:01:49.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17272025-04-16T04:01:49.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
17282025-04-16T04:01:49.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17292025-04-16T04:01:49.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17302025-04-16T04:01:49.517ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
17312025-04-16T04:01:49.517ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 71.510701ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
17322025-04-16T04:01:49.517ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17332025-04-16T04:01:49.517ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
17342025-04-16T04:01:49.518ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 71.689695ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
17352025-04-16T04:01:49.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 87.689941ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
17362025-04-16T04:01:49.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.087789ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
17372025-04-16T04:01:49.558ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
17382025-04-16T04:01:49.558ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17392025-04-16T04:01:49.559ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c27ff256-0241-4f1f-babc-e9f56cbe6240", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17402025-04-16T04:01:49.559ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17412025-04-16T04:01:49.561ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17422025-04-16T04:01:49.647ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 201.575149ms iteration = 18 status = {"physical_disks_added":0}
17432025-04-16T04:01:49.669ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
17442025-04-16T04:01:49.669ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17452025-04-16T04:01:49.669ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b04f18d-d2fd-4804-b6ff-861032698bbe", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17462025-04-16T04:01:49.669ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17472025-04-16T04:01:49.671ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17482025-04-16T04:01:49.679ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
17492025-04-16T04:01:49.679ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17502025-04-16T04:01:49.679ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff318b94-adc9-42b8-abaa-daa03b5c6f7e", "content-length": "372", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17512025-04-16T04:01:49.679ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17522025-04-16T04:01:49.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17532025-04-16T04:01:49.681ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 235.190493ms iteration = 16 status = 0
17542025-04-16T04:01:49.683ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
17552025-04-16T04:01:49.683ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17562025-04-16T04:01:49.683ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7bd0e9e-d171-4a84-95f0-73d45c3f55d9", "content-length": "26", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17572025-04-16T04:01:49.683ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17582025-04-16T04:01:49.685ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
17592025-04-16T04:01:49.695ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
17602025-04-16T04:01:49.695ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17612025-04-16T04:01:49.695ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9e06e642-b46b-484c-bcb0-3cc7fbe576d5", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17622025-04-16T04:01:49.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17632025-04-16T04:01:49.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17642025-04-16T04:01:49.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17652025-04-16T04:01:49.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17662025-04-16T04:01:49.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17672025-04-16T04:01:49.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17682025-04-16T04:01:49.704ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17692025-04-16T04:01:49.704ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17702025-04-16T04:01:49.705ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "75099131-2582-43c6-bef2-ee5939af2209", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17712025-04-16T04:01:49.706ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 259.953511ms iteration = 16 status = {}
17722025-04-16T04:01:49.711ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17732025-04-16T04:01:49.711ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e69037d0-cb9a-4b26-8fac-1df884b01f48", "content-length": "691", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
17742025-04-16T04:01:49.711ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17752025-04-16T04:01:49.712ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17762025-04-16T04:01:49.712ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17772025-04-16T04:01:49.713ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 266.625163ms iteration = 16 status = {}
17782025-04-16T04:01:49.737ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 291.316401ms iteration = 16 status = {}
17792025-04-16T04:01:49.781ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17802025-04-16T04:01:49.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17812025-04-16T04:01:49.783ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 337.585114ms iteration = 16 status = null
17822025-04-16T04:01:49.826ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33576560-b51c-47c1-8e83-70222ce5d4d1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
17832025-04-16T04:01:49.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33576560-b51c-47c1-8e83-70222ce5d4d1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
17842025-04-16T04:01:49.828ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 33576560-b51c-47c1-8e83-70222ce5d4d1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
17852025-04-16T04:01:49.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17862025-04-16T04:01:49.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17872025-04-16T04:01:49.871ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
17882025-04-16T04:01:49.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17892025-04-16T04:01:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17902025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
17912025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
17922025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17932025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17942025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17952025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17962025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17972025-04-16T04:01:49.887ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17982025-04-16T04:01:49.888ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17992025-04-16T04:01:49.888ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
18002025-04-16T04:01:49.888ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
18012025-04-16T04:01:49.888ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18022025-04-16T04:01:49.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18032025-04-16T04:01:49.890ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
18042025-04-16T04:01:49.890ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18052025-04-16T04:01:49.992ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "26af906d-180e-4cbb-8ae7-b755d0115b6b", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
18062025-04-16T04:01:49.992ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
18072025-04-16T04:01:49.992ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18082025-04-16T04:01:49.992ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b64ffcb5-222c-4e62-90a3-b25252f139b5", "content-length": "682", "date": "Wed, 16 Apr 2025 04:01:49 GMT"} })
18092025-04-16T04:01:49.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18102025-04-16T04:01:49.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18112025-04-16T04:01:49.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18122025-04-16T04:01:49.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18132025-04-16T04:01:49.998ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18142025-04-16T04:01:49.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18152025-04-16T04:01:50.000ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18162025-04-16T04:01:50.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18172025-04-16T04:01:50.008ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 561.583265ms iteration = 16 status = {}
18182025-04-16T04:01:50.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2211ab22-d7e4-46ae-9123-120f36633a1f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18192025-04-16T04:01:50.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2211ab22-d7e4-46ae-9123-120f36633a1f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18202025-04-16T04:01:50.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2329 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2211ab22-d7e4-46ae-9123-120f36633a1f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18212025-04-16T04:01:51.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84f51c86-aa40-4d52-a70b-45e8b7300fb3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18222025-04-16T04:01:51.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84f51c86-aa40-4d52-a70b-45e8b7300fb3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18232025-04-16T04:01:51.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2040 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 84f51c86-aa40-4d52-a70b-45e8b7300fb3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18242025-04-16T04:01:52.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = db47a41d-193c-42d1-85f9-68c3968ccfe1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18252025-04-16T04:01:52.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = db47a41d-193c-42d1-85f9-68c3968ccfe1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18262025-04-16T04:01:52.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2348 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = db47a41d-193c-42d1-85f9-68c3968ccfe1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18272025-04-16T04:01:53.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a84f0ea-4701-4f52-8909-f9dbc774ec03 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18282025-04-16T04:01:53.934ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a84f0ea-4701-4f52-8909-f9dbc774ec03 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18292025-04-16T04:01:53.935ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1829 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4a84f0ea-4701-4f52-8909-f9dbc774ec03 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18302025-04-16T04:01:54.956ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 173c7dd6-59b3-406c-9921-620afe912964 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18312025-04-16T04:01:54.956ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 173c7dd6-59b3-406c-9921-620afe912964 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18322025-04-16T04:01:54.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2293 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 173c7dd6-59b3-406c-9921-620afe912964 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18332025-04-16T04:01:55.977ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ba244c46-f350-4a9f-addb-929a7a3d81a3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18342025-04-16T04:01:55.977ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ba244c46-f350-4a9f-addb-929a7a3d81a3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18352025-04-16T04:01:55.979ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2399 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ba244c46-f350-4a9f-addb-929a7a3d81a3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18362025-04-16T04:01:57.000ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2929e6a9-b791-4857-abc8-b62955a2f2f3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18372025-04-16T04:01:57.000ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2929e6a9-b791-4857-abc8-b62955a2f2f3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18382025-04-16T04:01:57.002ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2021 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2929e6a9-b791-4857-abc8-b62955a2f2f3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18392025-04-16T04:01:58.022ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f67e369a-4332-47a3-885e-c2e3998b013e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18402025-04-16T04:01:58.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f67e369a-4332-47a3-885e-c2e3998b013e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18412025-04-16T04:01:58.024ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2791 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f67e369a-4332-47a3-885e-c2e3998b013e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18422025-04-16T04:01:58.565ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 526 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 9f837446-40df-459c-a28f-674eb4b72655 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
18432025-04-16T04:01:59.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1005f6e6-3ad3-48ea-bf97-aa05a729d88e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18442025-04-16T04:01:59.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1005f6e6-3ad3-48ea-bf97-aa05a729d88e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18452025-04-16T04:01:59.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1005f6e6-3ad3-48ea-bf97-aa05a729d88e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18462025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
18472025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
18482025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18492025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18502025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18512025-04-16T04:01:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18522025-04-16T04:01:59.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18532025-04-16T04:01:59.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18542025-04-16T04:01:59.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18552025-04-16T04:01:59.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.340396ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18562025-04-16T04:01:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18572025-04-16T04:01:59.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18582025-04-16T04:01:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18592025-04-16T04:01:59.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18602025-04-16T04:01:59.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18612025-04-16T04:01:59.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18622025-04-16T04:01:59.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
18632025-04-16T04:01:59.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18642025-04-16T04:01:59.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18652025-04-16T04:01:59.464ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
18662025-04-16T04:01:59.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.523626ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
18672025-04-16T04:02:00.066ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bab863cf-4eb5-4f2a-b7bc-3acf9efc5cb8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18682025-04-16T04:02:00.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bab863cf-4eb5-4f2a-b7bc-3acf9efc5cb8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18692025-04-16T04:02:00.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1736 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bab863cf-4eb5-4f2a-b7bc-3acf9efc5cb8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18702025-04-16T04:02:01.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03ca515f-2e78-47f0-bde9-3c9a7794dd2c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18712025-04-16T04:02:01.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03ca515f-2e78-47f0-bde9-3c9a7794dd2c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18722025-04-16T04:02:01.091ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03ca515f-2e78-47f0-bde9-3c9a7794dd2c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18732025-04-16T04:02:02.111ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c0632b4-388e-4b80-80f3-0f26a449adbf roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18742025-04-16T04:02:02.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c0632b4-388e-4b80-80f3-0f26a449adbf resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18752025-04-16T04:02:02.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2542 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c0632b4-388e-4b80-80f3-0f26a449adbf response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18762025-04-16T04:02:03.131ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 572b8847-05b2-4446-ad04-b0965b774dbd roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18772025-04-16T04:02:03.132ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 572b8847-05b2-4446-ad04-b0965b774dbd resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18782025-04-16T04:02:03.133ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1860 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 572b8847-05b2-4446-ad04-b0965b774dbd response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18792025-04-16T04:02:04.152ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3e6ca7c-532c-4123-bf2e-b792a34e6a1c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18802025-04-16T04:02:04.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3e6ca7c-532c-4123-bf2e-b792a34e6a1c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18812025-04-16T04:02:04.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2615 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3e6ca7c-532c-4123-bf2e-b792a34e6a1c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18822025-04-16T04:02:05.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd76e890-70a1-4ccd-a7bf-a9e4ca64ddfb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18832025-04-16T04:02:05.176ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd76e890-70a1-4ccd-a7bf-a9e4ca64ddfb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18842025-04-16T04:02:05.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2657 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd76e890-70a1-4ccd-a7bf-a9e4ca64ddfb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18852025-04-16T04:02:06.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16939549-6bcc-4e23-be4d-4cb2e266a033 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18862025-04-16T04:02:06.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16939549-6bcc-4e23-be4d-4cb2e266a033 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18872025-04-16T04:02:06.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2536 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 16939549-6bcc-4e23-be4d-4cb2e266a033 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18882025-04-16T04:02:07.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e859f6a-0a21-4f5d-a437-bfdf54c132b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18892025-04-16T04:02:07.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e859f6a-0a21-4f5d-a437-bfdf54c132b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18902025-04-16T04:02:07.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2362 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e859f6a-0a21-4f5d-a437-bfdf54c132b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18912025-04-16T04:02:08.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 07651b19-97d7-46d5-a23a-9cded19f31ae roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18922025-04-16T04:02:08.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 07651b19-97d7-46d5-a23a-9cded19f31ae resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18932025-04-16T04:02:08.244ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1875 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 07651b19-97d7-46d5-a23a-9cded19f31ae response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18942025-04-16T04:02:08.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = bace00b6-cbcd-4397-b6d4-618e79f2b3f1 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
18952025-04-16T04:02:09.265ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45778fa1-6258-44c2-a379-d5d00adac3f9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18962025-04-16T04:02:09.265ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45778fa1-6258-44c2-a379-d5d00adac3f9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18972025-04-16T04:02:09.267ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2213 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45778fa1-6258-44c2-a379-d5d00adac3f9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
18982025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
18992025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
19002025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19012025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19022025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19032025-04-16T04:02:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19042025-04-16T04:02:09.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19052025-04-16T04:02:09.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19062025-04-16T04:02:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19072025-04-16T04:02:09.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.963499ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19082025-04-16T04:02:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19092025-04-16T04:02:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19102025-04-16T04:02:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19112025-04-16T04:02:09.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19122025-04-16T04:02:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19132025-04-16T04:02:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19142025-04-16T04:02:09.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
19152025-04-16T04:02:09.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19162025-04-16T04:02:09.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19172025-04-16T04:02:09.464ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
19182025-04-16T04:02:09.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.039316ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
19192025-04-16T04:02:10.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70f49867-9520-4134-9a1d-7c585b074645 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19202025-04-16T04:02:10.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70f49867-9520-4134-9a1d-7c585b074645 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19212025-04-16T04:02:10.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2329 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70f49867-9520-4134-9a1d-7c585b074645 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19222025-04-16T04:02:11.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30db44bf-ffce-45d8-9890-47ace6bb9fe3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19232025-04-16T04:02:11.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30db44bf-ffce-45d8-9890-47ace6bb9fe3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19242025-04-16T04:02:11.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2147 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 30db44bf-ffce-45d8-9890-47ace6bb9fe3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19252025-04-16T04:02:12.330ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38fc59e6-9131-4f6a-8c49-9c540836f0d1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19262025-04-16T04:02:12.330ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38fc59e6-9131-4f6a-8c49-9c540836f0d1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19272025-04-16T04:02:12.332ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2502 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38fc59e6-9131-4f6a-8c49-9c540836f0d1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19282025-04-16T04:02:13.353ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70a805fd-d7e6-4f1b-bb11-2b1557633092 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19292025-04-16T04:02:13.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70a805fd-d7e6-4f1b-bb11-2b1557633092 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19302025-04-16T04:02:13.357ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 70a805fd-d7e6-4f1b-bb11-2b1557633092 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19312025-04-16T04:02:14.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97b8b515-cc20-4f89-832e-49e8946b1f8c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19322025-04-16T04:02:14.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97b8b515-cc20-4f89-832e-49e8946b1f8c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19332025-04-16T04:02:14.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97b8b515-cc20-4f89-832e-49e8946b1f8c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19342025-04-16T04:02:15.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 777edaaf-1f6f-49cb-ac40-b93ea5f42168 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19352025-04-16T04:02:15.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 777edaaf-1f6f-49cb-ac40-b93ea5f42168 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19362025-04-16T04:02:15.409ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1976 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 777edaaf-1f6f-49cb-ac40-b93ea5f42168 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19372025-04-16T04:02:16.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 592745f1-2e61-4aeb-aa07-3e837c1e4433 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19382025-04-16T04:02:16.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 592745f1-2e61-4aeb-aa07-3e837c1e4433 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19392025-04-16T04:02:16.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 592745f1-2e61-4aeb-aa07-3e837c1e4433 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19402025-04-16T04:02:17.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e520913c-6e20-4f96-96c3-19f275bb71e4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19412025-04-16T04:02:17.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e520913c-6e20-4f96-96c3-19f275bb71e4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19422025-04-16T04:02:17.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2089 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e520913c-6e20-4f96-96c3-19f275bb71e4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19432025-04-16T04:02:18.473ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bf111960-0c46-47c6-9c29-ae0edcbb8ddb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19442025-04-16T04:02:18.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bf111960-0c46-47c6-9c29-ae0edcbb8ddb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19452025-04-16T04:02:18.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bf111960-0c46-47c6-9c29-ae0edcbb8ddb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
19462025-04-16T04:02:18.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 557 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = fd261d2f-4ab3-4932-b26c-deb9358459db response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
19472025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
19482025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
19492025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
19502025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19512025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
19522025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19532025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
19542025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19552025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
19562025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19572025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19582025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
19592025-04-16T04:02:19.446ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
19602025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.53\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
19612025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
19622025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
19632025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19642025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19652025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19662025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
19672025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
19682025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19692025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19702025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
19712025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
19722025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19732025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
19742025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
19752025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19762025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19772025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
19782025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19792025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
19802025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19812025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19822025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19832025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
19842025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
19852025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
19862025-04-16T04:02:19.446ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
19872025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19882025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
19892025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19902025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
19912025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19922025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
19932025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
19942025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
19952025-04-16T04:02:19.446ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:52:19.446785800 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
19962025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
19972025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
19982025-04-16T04:02:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19992025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20002025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20012025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
20022025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20032025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
20042025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
20052025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20062025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20072025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
20082025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20092025-04-16T04:02:19.446ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
20102025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20112025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
20122025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20132025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20142025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
20152025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20162025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
20172025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20182025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20192025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
20202025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20212025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
20222025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20232025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20242025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20252025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20262025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20272025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20282025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
20292025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20302025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20312025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20322025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20332025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
20342025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20352025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20362025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20372025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20382025-04-16T04:02:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20392025-04-16T04:02:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3029, tv_nsec: 728551145 } })
20402025-04-16T04:02:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3029, tv_nsec: 728573675 } })
20412025-04-16T04:02:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
20422025-04-16T04:02:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
20432025-04-16T04:02:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.244775ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
20442025-04-16T04:02:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.649053ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
20452025-04-16T04:02:19.448ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20462025-04-16T04:02:19.449ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20472025-04-16T04:02:19.449ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3029, tv_nsec: 730798629 } })
20482025-04-16T04:02:19.450ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
20492025-04-16T04:02:19.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20502025-04-16T04:02:19.450ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20512025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20522025-04-16T04:02:19.451ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20532025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20542025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20552025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20562025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20572025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20582025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20592025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20602025-04-16T04:02:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20612025-04-16T04:02:19.452ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.249524ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20622025-04-16T04:02:19.453ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20632025-04-16T04:02:19.454ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20642025-04-16T04:02:19.454ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20652025-04-16T04:02:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20662025-04-16T04:02:19.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20672025-04-16T04:02:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20682025-04-16T04:02:19.456ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20692025-04-16T04:02:19.457ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20702025-04-16T04:02:19.457ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712025-04-16T04:02:19.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20722025-04-16T04:02:19.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20732025-04-16T04:02:19.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20742025-04-16T04:02:19.459ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:52:19.446785800 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
20752025-04-16T04:02:19.459ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.332173ms iteration = 9 status = {"expiration":"2025-04-16T03:52:19.446785800Z","pruned":[]}
20762025-04-16T04:02:19.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3029, tv_nsec: 741565332 } })
20772025-04-16T04:02:19.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20782025-04-16T04:02:19.460ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20792025-04-16T04:02:19.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
20802025-04-16T04:02:19.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20812025-04-16T04:02:19.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20822025-04-16T04:02:19.461ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20832025-04-16T04:02:19.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20842025-04-16T04:02:19.465ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.412494ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
20852025-04-16T04:02:19.465ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20862025-04-16T04:02:19.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20872025-04-16T04:02:19.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20882025-04-16T04:02:19.468ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20892025-04-16T04:02:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20902025-04-16T04:02:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20912025-04-16T04:02:19.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20922025-04-16T04:02:19.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20932025-04-16T04:02:19.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20942025-04-16T04:02:19.473ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20952025-04-16T04:02:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20962025-04-16T04:02:19.473ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20972025-04-16T04:02:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20982025-04-16T04:02:19.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20992025-04-16T04:02:19.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21002025-04-16T04:02:19.480ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21012025-04-16T04:02:19.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21022025-04-16T04:02:19.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21032025-04-16T04:02:19.482ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21042025-04-16T04:02:19.482ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21052025-04-16T04:02:19.482ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21062025-04-16T04:02:19.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21072025-04-16T04:02:19.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21082025-04-16T04:02:19.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21092025-04-16T04:02:19.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21102025-04-16T04:02:19.490ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21112025-04-16T04:02:19.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21122025-04-16T04:02:19.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21132025-04-16T04:02:19.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21142025-04-16T04:02:19.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21152025-04-16T04:02:19.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21162025-04-16T04:02:19.493ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
21172025-04-16T04:02:19.493ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 46.861596ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
21182025-04-16T04:02:19.494ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21192025-04-16T04:02:19.494ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21202025-04-16T04:02:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21212025-04-16T04:02:19.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7c9e192e-fcc6-482c-bc26-059473971b9b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
21222025-04-16T04:02:19.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7c9e192e-fcc6-482c-bc26-059473971b9b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
21232025-04-16T04:02:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21242025-04-16T04:02:19.497ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21252025-04-16T04:02:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21262025-04-16T04:02:19.498ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 51.787732ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
21272025-04-16T04:02:19.499ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.109748ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
21282025-04-16T04:02:19.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21292025-04-16T04:02:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21302025-04-16T04:02:19.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21312025-04-16T04:02:19.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21322025-04-16T04:02:19.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21332025-04-16T04:02:19.509ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 63.125986ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
21342025-04-16T04:02:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21352025-04-16T04:02:19.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21362025-04-16T04:02:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21372025-04-16T04:02:19.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21382025-04-16T04:02:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21392025-04-16T04:02:19.516ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21402025-04-16T04:02:19.516ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
21412025-04-16T04:02:19.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21422025-04-16T04:02:19.516ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21432025-04-16T04:02:19.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21442025-04-16T04:02:19.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21452025-04-16T04:02:19.519ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
21462025-04-16T04:02:19.519ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
21472025-04-16T04:02:19.519ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 72.487651ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
21482025-04-16T04:02:19.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21492025-04-16T04:02:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21502025-04-16T04:02:19.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
21512025-04-16T04:02:19.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
21522025-04-16T04:02:19.535ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21532025-04-16T04:02:19.535ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
21542025-04-16T04:02:19.535ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 88.405875ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21552025-04-16T04:02:19.562ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21562025-04-16T04:02:19.562ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21572025-04-16T04:02:19.562ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21582025-04-16T04:02:19.562ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21592025-04-16T04:02:19.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21602025-04-16T04:02:19.613ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
21612025-04-16T04:02:19.613ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21622025-04-16T04:02:19.613ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fba9618-4df1-4c9a-ab1a-557fa25929ca", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
21632025-04-16T04:02:19.613ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21642025-04-16T04:02:19.616ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21652025-04-16T04:02:19.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21662025-04-16T04:02:19.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21672025-04-16T04:02:19.693ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
21682025-04-16T04:02:19.693ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21692025-04-16T04:02:19.693ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da63c8c-e669-485b-b2e3-3f87081a069e", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
21702025-04-16T04:02:19.693ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21712025-04-16T04:02:19.695ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21722025-04-16T04:02:19.696ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
21732025-04-16T04:02:19.734ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
21742025-04-16T04:02:19.734ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21752025-04-16T04:02:19.734ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8979371-f888-4cb2-aeb0-528412640218", "content-length": "372", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
21762025-04-16T04:02:19.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21772025-04-16T04:02:19.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21782025-04-16T04:02:19.736ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21792025-04-16T04:02:19.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21802025-04-16T04:02:19.739ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
21812025-04-16T04:02:19.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21822025-04-16T04:02:19.739ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21832025-04-16T04:02:19.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21842025-04-16T04:02:19.741ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
21852025-04-16T04:02:19.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21862025-04-16T04:02:19.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21872025-04-16T04:02:19.742ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21882025-04-16T04:02:19.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21892025-04-16T04:02:19.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 246009 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7c9e192e-fcc6-482c-bc26-059473971b9b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
21902025-04-16T04:02:19.744ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21912025-04-16T04:02:19.745ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3030, tv_nsec: 26936565 } })
21922025-04-16T04:02:19.746ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
21932025-04-16T04:02:19.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
21942025-04-16T04:02:19.746ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21952025-04-16T04:02:19.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21962025-04-16T04:02:19.761ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
21972025-04-16T04:02:19.761ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 314.748523ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
21982025-04-16T04:02:19.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21992025-04-16T04:02:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22002025-04-16T04:02:19.768ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22012025-04-16T04:02:19.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22022025-04-16T04:02:19.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22032025-04-16T04:02:19.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
22042025-04-16T04:02:19.775ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22052025-04-16T04:02:19.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22062025-04-16T04:02:19.777ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 330.784859ms iteration = 17 status = 0
22072025-04-16T04:02:19.786ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
22082025-04-16T04:02:19.786ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 340.488901ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
22092025-04-16T04:02:19.791ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 344.691714ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
22102025-04-16T04:02:19.797ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22112025-04-16T04:02:19.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22122025-04-16T04:02:19.798ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22132025-04-16T04:02:19.798ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
22142025-04-16T04:02:19.798ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 352.280794ms iteration = 19 status = {"physical_disks_added":0}
22152025-04-16T04:02:19.798ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 352.274814ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
22162025-04-16T04:02:19.798ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 351.822865ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
22172025-04-16T04:02:19.799ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22182025-04-16T04:02:19.801ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 355.034417ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22192025-04-16T04:02:19.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22202025-04-16T04:02:19.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22212025-04-16T04:02:19.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22222025-04-16T04:02:19.803ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22232025-04-16T04:02:19.803ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22242025-04-16T04:02:19.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22252025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22262025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22272025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22282025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22292025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22302025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22312025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
22322025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22332025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22342025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22352025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22362025-04-16T04:02:19.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22372025-04-16T04:02:19.808ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 362.085647ms iteration = 17 status = {}
22382025-04-16T04:02:19.809ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22392025-04-16T04:02:19.809ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22402025-04-16T04:02:19.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22412025-04-16T04:02:19.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22422025-04-16T04:02:19.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22432025-04-16T04:02:19.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22442025-04-16T04:02:19.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22452025-04-16T04:02:19.818ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
22462025-04-16T04:02:19.818ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22472025-04-16T04:02:19.818ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
22482025-04-16T04:02:19.818ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 371.761988ms iteration = 9 status = {"generation":2}
22492025-04-16T04:02:19.827ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22502025-04-16T04:02:19.827ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22512025-04-16T04:02:19.827ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d7279d41-d4d5-4d4b-9b2f-1cdc20562864", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
22522025-04-16T04:02:19.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22532025-04-16T04:02:19.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22542025-04-16T04:02:19.839ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22552025-04-16T04:02:19.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22562025-04-16T04:02:19.848ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22572025-04-16T04:02:19.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22582025-04-16T04:02:19.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22592025-04-16T04:02:19.854ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22602025-04-16T04:02:19.854ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22612025-04-16T04:02:19.856ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22622025-04-16T04:02:19.856ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22632025-04-16T04:02:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22642025-04-16T04:02:19.858ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f54471b0-bd00-4607-92e5-ad47df703215", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
22652025-04-16T04:02:19.860ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 413.913469ms iteration = 17 status = {}
22662025-04-16T04:02:19.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22672025-04-16T04:02:19.867ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
22682025-04-16T04:02:19.867ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 420.52395ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
22692025-04-16T04:02:19.870ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22702025-04-16T04:02:19.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22712025-04-16T04:02:19.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22722025-04-16T04:02:19.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22732025-04-16T04:02:19.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22742025-04-16T04:02:19.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 424.470147ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
22752025-04-16T04:02:19.873ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 426.849375ms iteration = 17 status = null
22762025-04-16T04:02:19.874ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
22772025-04-16T04:02:19.874ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22782025-04-16T04:02:19.874ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
22792025-04-16T04:02:19.874ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 427.947536ms iteration = 9 status = {"generation":1}
22802025-04-16T04:02:19.881ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22812025-04-16T04:02:19.881ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e553c052-5bc2-4ec4-b8da-4c217d3d6d7c", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22822025-04-16T04:02:19.881ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22832025-04-16T04:02:19.883ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ea5b01c6-ec67-4aa4-b43c-2984848913d1", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22842025-04-16T04:02:19.883ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22852025-04-16T04:02:19.885ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
22862025-04-16T04:02:19.885ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 438.550915ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
22872025-04-16T04:02:19.885ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22882025-04-16T04:02:19.885ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 438.871552ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22892025-04-16T04:02:19.904ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
22902025-04-16T04:02:19.904ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22912025-04-16T04:02:19.905ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c15219a-f531-4c32-ba6d-89dee87f510b", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
22922025-04-16T04:02:19.905ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22932025-04-16T04:02:19.906ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
22942025-04-16T04:02:19.924ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22952025-04-16T04:02:20.010ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22962025-04-16T04:02:20.011ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b0c4b63b-5837-4200-bb75-5b90058a05ea", "date": "Wed, 16 Apr 2025 04:02:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22972025-04-16T04:02:20.011ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22982025-04-16T04:02:20.013ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2a595c62-abe1-483a-ad00-c986498339db", "date": "Wed, 16 Apr 2025 04:02:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
22992025-04-16T04:02:20.013ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
23002025-04-16T04:02:20.015ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "aa67d173-2803-4532-9dcd-9b516c86a904", "date": "Wed, 16 Apr 2025 04:02:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
23012025-04-16T04:02:20.015ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
23022025-04-16T04:02:20.017ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
23032025-04-16T04:02:20.017ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 571.403998ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
23042025-04-16T04:02:20.023ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23052025-04-16T04:02:20.023ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "20c3bbd0-8e6c-4644-99b8-dc98fce3087b", "content-length": "691", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
23062025-04-16T04:02:20.023ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
23072025-04-16T04:02:20.023ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
23082025-04-16T04:02:20.023ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
23092025-04-16T04:02:20.025ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 578.428996ms iteration = 17 status = {}
23102025-04-16T04:02:20.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23112025-04-16T04:02:20.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23122025-04-16T04:02:20.103ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
23132025-04-16T04:02:20.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23142025-04-16T04:02:20.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23152025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
23162025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
23172025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
23182025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23192025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23202025-04-16T04:02:20.115ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23212025-04-16T04:02:20.116ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23222025-04-16T04:02:20.116ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23232025-04-16T04:02:20.116ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23242025-04-16T04:02:20.116ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23252025-04-16T04:02:20.116ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
23262025-04-16T04:02:20.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23272025-04-16T04:02:20.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23282025-04-16T04:02:20.119ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
23292025-04-16T04:02:20.119ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23302025-04-16T04:02:20.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7ce58069-656b-469e-8695-e691169bc4b4", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
23312025-04-16T04:02:20.210ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
23322025-04-16T04:02:20.210ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23332025-04-16T04:02:20.214ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ee95e3d-468d-403b-8f76-32e1c69be8ff", "content-length": "682", "date": "Wed, 16 Apr 2025 04:02:19 GMT"} })
23342025-04-16T04:02:20.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23352025-04-16T04:02:20.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23362025-04-16T04:02:20.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23372025-04-16T04:02:20.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23382025-04-16T04:02:20.221ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23392025-04-16T04:02:20.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23402025-04-16T04:02:20.224ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23412025-04-16T04:02:20.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23422025-04-16T04:02:20.232ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 785.513354ms iteration = 17 status = {}
23432025-04-16T04:02:20.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1adc55a2-ef6b-403e-a92c-451a1b68af5a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23442025-04-16T04:02:20.767ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1adc55a2-ef6b-403e-a92c-451a1b68af5a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23452025-04-16T04:02:20.769ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1adc55a2-ef6b-403e-a92c-451a1b68af5a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23462025-04-16T04:02:21.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8fe2ad44-fb84-4846-a6d8-b05b4e325756 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23472025-04-16T04:02:21.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8fe2ad44-fb84-4846-a6d8-b05b4e325756 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23482025-04-16T04:02:21.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2174 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8fe2ad44-fb84-4846-a6d8-b05b4e325756 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23492025-04-16T04:02:22.810ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a5a96f0-68b2-4aa9-83da-18f6f773ac7e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23502025-04-16T04:02:22.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a5a96f0-68b2-4aa9-83da-18f6f773ac7e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23512025-04-16T04:02:22.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a5a96f0-68b2-4aa9-83da-18f6f773ac7e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23522025-04-16T04:02:23.833ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 21d4fa6e-3821-49dd-a3eb-d00d034af799 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23532025-04-16T04:02:23.833ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 21d4fa6e-3821-49dd-a3eb-d00d034af799 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23542025-04-16T04:02:23.835ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 21d4fa6e-3821-49dd-a3eb-d00d034af799 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23552025-04-16T04:02:24.855ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7910a47-ec61-4851-9ee8-07b269630905 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23562025-04-16T04:02:24.855ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7910a47-ec61-4851-9ee8-07b269630905 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23572025-04-16T04:02:24.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4652 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7910a47-ec61-4851-9ee8-07b269630905 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23582025-04-16T04:02:25.880ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc53512d-745e-4e66-9458-ede97b38ea60 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23592025-04-16T04:02:25.880ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc53512d-745e-4e66-9458-ede97b38ea60 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23602025-04-16T04:02:25.882ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2108 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc53512d-745e-4e66-9458-ede97b38ea60 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23612025-04-16T04:02:26.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e62d7ef3-03e3-495a-9108-d4f0d9544a8b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23622025-04-16T04:02:26.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e62d7ef3-03e3-495a-9108-d4f0d9544a8b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23632025-04-16T04:02:26.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2142 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e62d7ef3-03e3-495a-9108-d4f0d9544a8b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23642025-04-16T04:02:27.926ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e16e48f5-02bf-4d7f-b35a-6a2727ca4e19 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23652025-04-16T04:02:27.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e16e48f5-02bf-4d7f-b35a-6a2727ca4e19 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23662025-04-16T04:02:27.928ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e16e48f5-02bf-4d7f-b35a-6a2727ca4e19 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23672025-04-16T04:02:28.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 526 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 45cba11c-f894-4789-9314-2252b2e29032 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
23682025-04-16T04:02:28.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7bf51ff3-aa3d-46da-a681-4196caddff06 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23692025-04-16T04:02:28.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7bf51ff3-aa3d-46da-a681-4196caddff06 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23702025-04-16T04:02:28.950ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1808 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7bf51ff3-aa3d-46da-a681-4196caddff06 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23712025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
23722025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
23732025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23742025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23752025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23762025-04-16T04:02:29.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23772025-04-16T04:02:29.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23782025-04-16T04:02:29.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23792025-04-16T04:02:29.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23802025-04-16T04:02:29.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.350566ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23812025-04-16T04:02:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23822025-04-16T04:02:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23832025-04-16T04:02:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23842025-04-16T04:02:29.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23852025-04-16T04:02:29.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23862025-04-16T04:02:29.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23872025-04-16T04:02:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
23882025-04-16T04:02:29.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23892025-04-16T04:02:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23902025-04-16T04:02:29.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
23912025-04-16T04:02:29.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.939823ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
23922025-04-16T04:02:29.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c92798b-11f3-4990-9fbe-6029b1de079c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23932025-04-16T04:02:29.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c92798b-11f3-4990-9fbe-6029b1de079c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23942025-04-16T04:02:29.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1977 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8c92798b-11f3-4990-9fbe-6029b1de079c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23952025-04-16T04:02:30.993ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f89f8008-0b9e-4129-ba91-c7397d0f0d37 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23962025-04-16T04:02:30.993ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f89f8008-0b9e-4129-ba91-c7397d0f0d37 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23972025-04-16T04:02:30.995ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1906 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f89f8008-0b9e-4129-ba91-c7397d0f0d37 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23982025-04-16T04:02:32.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ded3c9a8-9ffc-4c03-9c3a-5adc9878943c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
23992025-04-16T04:02:32.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ded3c9a8-9ffc-4c03-9c3a-5adc9878943c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24002025-04-16T04:02:32.017ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ded3c9a8-9ffc-4c03-9c3a-5adc9878943c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24012025-04-16T04:02:33.038ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e224c54-53ef-4561-b73e-f4046a7ce825 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24022025-04-16T04:02:33.038ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e224c54-53ef-4561-b73e-f4046a7ce825 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24032025-04-16T04:02:33.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1807 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e224c54-53ef-4561-b73e-f4046a7ce825 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24042025-04-16T04:02:34.059ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c25a11e-c6fc-45e3-8fd0-2f96369f8e5e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24052025-04-16T04:02:34.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c25a11e-c6fc-45e3-8fd0-2f96369f8e5e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24062025-04-16T04:02:34.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2899 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c25a11e-c6fc-45e3-8fd0-2f96369f8e5e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24072025-04-16T04:02:35.083ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02c7cc71-3d9b-4a10-b2b5-c40f36f09e38 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24082025-04-16T04:02:35.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02c7cc71-3d9b-4a10-b2b5-c40f36f09e38 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24092025-04-16T04:02:35.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02c7cc71-3d9b-4a10-b2b5-c40f36f09e38 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24102025-04-16T04:02:36.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 115d0dfa-5950-47c3-bf7f-083b973ac489 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24112025-04-16T04:02:36.104ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 115d0dfa-5950-47c3-bf7f-083b973ac489 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24122025-04-16T04:02:36.106ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2218 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 115d0dfa-5950-47c3-bf7f-083b973ac489 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24132025-04-16T04:02:37.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5fba7216-9d5f-47da-bb11-2005345f8c85 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24142025-04-16T04:02:37.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5fba7216-9d5f-47da-bb11-2005345f8c85 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24152025-04-16T04:02:37.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2326 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5fba7216-9d5f-47da-bb11-2005345f8c85 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24162025-04-16T04:02:38.148ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7f3ad54-aaae-486c-b418-73fda316fd16 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24172025-04-16T04:02:38.149ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7f3ad54-aaae-486c-b418-73fda316fd16 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24182025-04-16T04:02:38.150ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1894 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e7f3ad54-aaae-486c-b418-73fda316fd16 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24192025-04-16T04:02:38.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 466 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = d2f49af9-0771-4b43-90e3-0decb774f989 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
24202025-04-16T04:02:39.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42215f3a-ecef-4177-8a86-73841c920404 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24212025-04-16T04:02:39.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42215f3a-ecef-4177-8a86-73841c920404 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24222025-04-16T04:02:39.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42215f3a-ecef-4177-8a86-73841c920404 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24232025-04-16T04:02:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
24242025-04-16T04:02:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24252025-04-16T04:02:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
24262025-04-16T04:02:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24272025-04-16T04:02:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24282025-04-16T04:02:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24292025-04-16T04:02:39.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24302025-04-16T04:02:39.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24312025-04-16T04:02:39.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24322025-04-16T04:02:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24332025-04-16T04:02:39.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24342025-04-16T04:02:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24352025-04-16T04:02:39.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24362025-04-16T04:02:39.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24372025-04-16T04:02:39.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24382025-04-16T04:02:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
24392025-04-16T04:02:39.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24402025-04-16T04:02:39.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24412025-04-16T04:02:39.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
24422025-04-16T04:02:39.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.703269ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
24432025-04-16T04:02:39.485ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.763155ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24442025-04-16T04:02:40.191ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 490863fc-4b5d-43a7-b4b9-926baaada267 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24452025-04-16T04:02:40.191ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 490863fc-4b5d-43a7-b4b9-926baaada267 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24462025-04-16T04:02:40.193ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2325 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 490863fc-4b5d-43a7-b4b9-926baaada267 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24472025-04-16T04:02:41.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f0079c9-5620-4d06-9641-b921ea534571 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24482025-04-16T04:02:41.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f0079c9-5620-4d06-9641-b921ea534571 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24492025-04-16T04:02:41.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1967 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f0079c9-5620-4d06-9641-b921ea534571 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24502025-04-16T04:02:42.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cfaef037-7096-4ee3-9527-77caecbca287 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24512025-04-16T04:02:42.235ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cfaef037-7096-4ee3-9527-77caecbca287 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24522025-04-16T04:02:42.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1994 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cfaef037-7096-4ee3-9527-77caecbca287 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24532025-04-16T04:02:43.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c6b496cc-ebe7-47f1-9150-e4bb7aecd272 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24542025-04-16T04:02:43.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c6b496cc-ebe7-47f1-9150-e4bb7aecd272 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24552025-04-16T04:02:43.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2419 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c6b496cc-ebe7-47f1-9150-e4bb7aecd272 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24562025-04-16T04:02:43.557ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:58028
24572025-04-16T04:02:43.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = a209e95d-47c4-40ef-abed-bd7e0e2eaf8f roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
24582025-04-16T04:02:43.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = a209e95d-47c4-40ef-abed-bd7e0e2eaf8f resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
24592025-04-16T04:02:43.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 5999 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = a209e95d-47c4-40ef-abed-bd7e0e2eaf8f response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
24602025-04-16T04:02:43.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = e34a69d9-c178-4fc0-9e67-4bea26e69004 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
24612025-04-16T04:02:43.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = e34a69d9-c178-4fc0-9e67-4bea26e69004 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
24622025-04-16T04:02:43.566ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2798 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = e34a69d9-c178-4fc0-9e67-4bea26e69004 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
24632025-04-16T04:02:44.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3615b576-b702-4011-9b7b-70d3e527585b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24642025-04-16T04:02:44.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3615b576-b702-4011-9b7b-70d3e527585b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24652025-04-16T04:02:44.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2114 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3615b576-b702-4011-9b7b-70d3e527585b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24662025-04-16T04:02:45.300ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2a04b00-b770-473f-9cab-3d8ffd37c169 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24672025-04-16T04:02:45.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2a04b00-b770-473f-9cab-3d8ffd37c169 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24682025-04-16T04:02:45.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e2a04b00-b770-473f-9cab-3d8ffd37c169 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24692025-04-16T04:02:46.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3fec48d-def4-4de3-b5a0-35296c262fe8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24702025-04-16T04:02:46.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3fec48d-def4-4de3-b5a0-35296c262fe8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24712025-04-16T04:02:46.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1994 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f3fec48d-def4-4de3-b5a0-35296c262fe8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24722025-04-16T04:02:47.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d1d896-63c5-4838-a0f0-3a4141fb78c2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24732025-04-16T04:02:47.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d1d896-63c5-4838-a0f0-3a4141fb78c2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24742025-04-16T04:02:47.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2043 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 98d1d896-63c5-4838-a0f0-3a4141fb78c2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24752025-04-16T04:02:48.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd0b5f9f-b510-42df-a0de-4b048a4119f7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24762025-04-16T04:02:48.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd0b5f9f-b510-42df-a0de-4b048a4119f7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24772025-04-16T04:02:48.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2085 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd0b5f9f-b510-42df-a0de-4b048a4119f7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24782025-04-16T04:02:48.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 483 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = e7ff6683-048e-4f0e-ab8a-5247349763f7 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
24792025-04-16T04:02:49.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cf9c01b5-b454-40df-8b95-7a8c2f23ca59 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24802025-04-16T04:02:49.391ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cf9c01b5-b454-40df-8b95-7a8c2f23ca59 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24812025-04-16T04:02:49.395ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4643 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cf9c01b5-b454-40df-8b95-7a8c2f23ca59 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
24822025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
24832025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
24842025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24852025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24862025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
24872025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24882025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
24892025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24902025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
24912025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
24922025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24932025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24942025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
24952025-04-16T04:02:49.445ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
24962025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
24972025-04-16T04:02:49.445ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24982025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
24992025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
25002025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
25012025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
25022025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25032025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
25042025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25052025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
25062025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
25072025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25082025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
25092025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25102025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
25112025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
25122025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
25132025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
25142025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
25152025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25162025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25172025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
25182025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25192025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
25202025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25212025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25222025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25232025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25242025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25252025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25262025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
25272025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25282025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25292025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
25302025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25312025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25322025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25332025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25342025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25352025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
25362025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25372025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25382025-04-16T04:02:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25392025-04-16T04:02:49.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25402025-04-16T04:02:49.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25412025-04-16T04:02:49.448ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25422025-04-16T04:02:49.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25432025-04-16T04:02:49.450ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.705784ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
25442025-04-16T04:02:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25452025-04-16T04:02:49.451ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25462025-04-16T04:02:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25472025-04-16T04:02:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25482025-04-16T04:02:49.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25492025-04-16T04:02:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25502025-04-16T04:02:49.454ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25512025-04-16T04:02:49.455ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25522025-04-16T04:02:49.455ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25532025-04-16T04:02:49.456ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3059, tv_nsec: 737457412 } })
25542025-04-16T04:02:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25552025-04-16T04:02:49.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
25562025-04-16T04:02:49.456ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25572025-04-16T04:02:49.456ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25582025-04-16T04:02:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25592025-04-16T04:02:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25602025-04-16T04:02:49.457ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 11.963206ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
25612025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25622025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25632025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25642025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25652025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.485276ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
25662025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
25672025-04-16T04:02:49.458ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25682025-04-16T04:02:49.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25692025-04-16T04:02:49.459ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25702025-04-16T04:02:49.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25712025-04-16T04:02:49.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25722025-04-16T04:02:49.460ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25732025-04-16T04:02:49.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25742025-04-16T04:02:49.461ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3059, tv_nsec: 742174445 } })
25752025-04-16T04:02:49.461ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
25762025-04-16T04:02:49.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25772025-04-16T04:02:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25782025-04-16T04:02:49.464ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3059, tv_nsec: 745219435 } })
25792025-04-16T04:02:49.464ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
25802025-04-16T04:02:49.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812025-04-16T04:02:49.467ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25822025-04-16T04:02:49.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25832025-04-16T04:02:49.468ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25842025-04-16T04:02:49.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25852025-04-16T04:02:49.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
25862025-04-16T04:02:49.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25872025-04-16T04:02:49.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25882025-04-16T04:02:49.475ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
25892025-04-16T04:02:49.475ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.35765ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
25902025-04-16T04:02:49.484ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 38.341456ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
25912025-04-16T04:02:49.488ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
25922025-04-16T04:02:49.488ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.964118ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
25932025-04-16T04:02:49.490ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25942025-04-16T04:02:49.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25952025-04-16T04:02:49.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25962025-04-16T04:02:49.501ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25972025-04-16T04:02:49.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25982025-04-16T04:02:49.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25992025-04-16T04:02:49.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26002025-04-16T04:02:49.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26012025-04-16T04:02:49.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26022025-04-16T04:02:49.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26032025-04-16T04:02:49.503ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 57.70934ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26042025-04-16T04:02:49.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26052025-04-16T04:02:49.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26062025-04-16T04:02:49.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26072025-04-16T04:02:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26082025-04-16T04:02:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26092025-04-16T04:02:49.506ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.242069ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26102025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26112025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26122025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26132025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26142025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26152025-04-16T04:02:49.507ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26162025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
26172025-04-16T04:02:49.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26182025-04-16T04:02:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26192025-04-16T04:02:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
26202025-04-16T04:02:49.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26212025-04-16T04:02:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26222025-04-16T04:02:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
26232025-04-16T04:02:49.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26242025-04-16T04:02:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26252025-04-16T04:02:49.513ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 67.166246ms iteration = 18 status = 0
26262025-04-16T04:02:49.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26272025-04-16T04:02:49.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26282025-04-16T04:02:49.516ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26292025-04-16T04:02:49.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26302025-04-16T04:02:49.521ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.730495ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26312025-04-16T04:02:49.531ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
26322025-04-16T04:02:49.531ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26332025-04-16T04:02:49.531ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0166a2c2-d04e-4bb7-b15c-7adde2514819", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:48 GMT"} })
26342025-04-16T04:02:49.531ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26352025-04-16T04:02:49.532ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26362025-04-16T04:02:49.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26372025-04-16T04:02:49.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26382025-04-16T04:02:49.537ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
26392025-04-16T04:02:49.537ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26402025-04-16T04:02:49.537ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "46636ab7-d0ad-4bf7-bf0f-002ff202504d", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:48 GMT"} })
26412025-04-16T04:02:49.537ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26422025-04-16T04:02:49.540ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26432025-04-16T04:02:49.541ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26442025-04-16T04:02:49.541ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
26452025-04-16T04:02:49.541ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26462025-04-16T04:02:49.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26472025-04-16T04:02:49.542ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
26482025-04-16T04:02:49.542ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 96.358152ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
26492025-04-16T04:02:49.543ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
26502025-04-16T04:02:49.543ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26512025-04-16T04:02:49.543ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6def4b2-566c-47b3-9a2f-e907d24ab2f3", "content-length": "26", "date": "Wed, 16 Apr 2025 04:02:48 GMT"} })
26522025-04-16T04:02:49.543ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26532025-04-16T04:02:49.545ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
26542025-04-16T04:02:49.555ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 109.773367ms iteration = 20 status = {"physical_disks_added":0}
26552025-04-16T04:02:49.585ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 139.212308ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
26562025-04-16T04:02:49.596ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26572025-04-16T04:02:49.596ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
26582025-04-16T04:02:49.596ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 150.527771ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
26592025-04-16T04:02:49.601ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26602025-04-16T04:02:49.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26612025-04-16T04:02:49.603ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26622025-04-16T04:02:49.605ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26632025-04-16T04:02:49.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26642025-04-16T04:02:49.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26652025-04-16T04:02:49.608ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26662025-04-16T04:02:49.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26672025-04-16T04:02:49.611ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26682025-04-16T04:02:49.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26692025-04-16T04:02:49.642ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 196.05961ms iteration = 18 status = {}
26702025-04-16T04:02:49.684ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26712025-04-16T04:02:49.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26722025-04-16T04:02:49.739ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26732025-04-16T04:02:49.740ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "92781f85-4462-4b85-b9ea-62bf2009be74", "content-length": "691", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
26742025-04-16T04:02:49.740ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
26752025-04-16T04:02:49.740ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26762025-04-16T04:02:49.740ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26772025-04-16T04:02:49.742ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 296.583495ms iteration = 18 status = {}
26782025-04-16T04:02:49.778ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
26792025-04-16T04:02:49.778ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26802025-04-16T04:02:49.778ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "69603a1a-c2fa-464b-9ce5-155547ccc651", "content-length": "372", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
26812025-04-16T04:02:49.778ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26822025-04-16T04:02:49.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26832025-04-16T04:02:49.780ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 334.512973ms iteration = 18 status = null
26842025-04-16T04:02:49.848ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26852025-04-16T04:02:49.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26862025-04-16T04:02:49.849ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
26872025-04-16T04:02:49.849ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
26882025-04-16T04:02:49.850ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ded8468d-7b9d-4168-9a0b-40f145e44d55", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
26892025-04-16T04:02:49.850ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26902025-04-16T04:02:49.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26912025-04-16T04:02:49.851ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
26922025-04-16T04:02:49.851ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26932025-04-16T04:02:49.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26942025-04-16T04:02:49.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26952025-04-16T04:02:49.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26962025-04-16T04:02:49.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26972025-04-16T04:02:49.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26982025-04-16T04:02:49.862ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
26992025-04-16T04:02:49.863ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27002025-04-16T04:02:49.863ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "51fcc27f-43fb-4f61-a328-375ed7bfc9c0", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
27012025-04-16T04:02:49.865ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 419.826246ms iteration = 18 status = {}
27022025-04-16T04:02:49.870ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
27032025-04-16T04:02:49.870ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
27042025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
27052025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27062025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27072025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27082025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27092025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27102025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27112025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27122025-04-16T04:02:49.871ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
27132025-04-16T04:02:49.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27142025-04-16T04:02:49.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27152025-04-16T04:02:49.874ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
27162025-04-16T04:02:49.874ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27172025-04-16T04:02:49.987ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "35c8956f-7c41-4359-952f-a811ff7f5ddd", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
27182025-04-16T04:02:49.987ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
27192025-04-16T04:02:49.987ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27202025-04-16T04:02:49.988ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ac62c16-1c24-461d-8ddb-250a46fd0aff", "content-length": "682", "date": "Wed, 16 Apr 2025 04:02:49 GMT"} })
27212025-04-16T04:02:49.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27222025-04-16T04:02:49.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27232025-04-16T04:02:49.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27242025-04-16T04:02:49.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27252025-04-16T04:02:49.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27262025-04-16T04:02:49.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27272025-04-16T04:02:49.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27282025-04-16T04:02:49.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27292025-04-16T04:02:50.004ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 558.704457ms iteration = 18 status = {}
27302025-04-16T04:02:50.416ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fead56b0-ca4c-443d-a84e-f870f8f2ae8a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27312025-04-16T04:02:50.416ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fead56b0-ca4c-443d-a84e-f870f8f2ae8a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27322025-04-16T04:02:50.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2352 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fead56b0-ca4c-443d-a84e-f870f8f2ae8a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27332025-04-16T04:02:50.566ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
27342025-04-16T04:02:50.566ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
27352025-04-16T04:02:50.597ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
27362025-04-16T04:02:50.597ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:45133
27372025-04-16T04:02:50.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45133 req_id = 8071542e-8deb-49b2-a4b3-3b40aef4fed0 roles = RoleSet { roles: {} } uri = /metrics/producers
27382025-04-16T04:02:50.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45133 req_id = 8071542e-8deb-49b2-a4b3-3b40aef4fed0 resource = Database result = Ok(()) uri = /metrics/producers
27392025-04-16T04:02:50.638ZINFOnexus: assigned collector to new producer collector_id = 0efaecbf-c4c4-456a-96b6-2cd10690be0f file = nexus/src/app/oximeter.rs:76 producer_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
27402025-04-16T04:02:50.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 41281 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:45133 req_id = 8071542e-8deb-49b2-a4b3-3b40aef4fed0 response_code = 201 uri = /metrics/producers
27412025-04-16T04:02:50.639ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "8071542e-8deb-49b2-a4b3-3b40aef4fed0", "content-length": "41", "date": "Wed, 16 Apr 2025 04:02:50 GMT"} })
27422025-04-16T04:02:50.641ZDEBGnexus (producer-registration-task): registered with nexus successfully
27432025-04-16T04:02:50.641ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
27442025-04-16T04:02:51.438ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 226ad1c6-8121-43e2-83dd-1e0c7823f1ff roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27452025-04-16T04:02:51.438ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 226ad1c6-8121-43e2-83dd-1e0c7823f1ff resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27462025-04-16T04:02:51.440ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2342 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 226ad1c6-8121-43e2-83dd-1e0c7823f1ff response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27472025-04-16T04:02:52.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebecb5a0-c949-4cb9-997f-1ff35d137782 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27482025-04-16T04:02:52.460ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebecb5a0-c949-4cb9-997f-1ff35d137782 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27492025-04-16T04:02:52.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2290 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebecb5a0-c949-4cb9-997f-1ff35d137782 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27502025-04-16T04:02:53.482ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02215afe-3c8f-4558-8317-7be6c7355db3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27512025-04-16T04:02:53.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02215afe-3c8f-4558-8317-7be6c7355db3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27522025-04-16T04:02:53.485ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3564 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02215afe-3c8f-4558-8317-7be6c7355db3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27532025-04-16T04:02:54.507ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7aa94251-8286-4949-8560-5841e7d97897 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27542025-04-16T04:02:54.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7aa94251-8286-4949-8560-5841e7d97897 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27552025-04-16T04:02:54.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2170 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7aa94251-8286-4949-8560-5841e7d97897 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27562025-04-16T04:02:55.529ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfe40c9c-7629-410d-bcf6-6adf60d7a6b2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27572025-04-16T04:02:55.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfe40c9c-7629-410d-bcf6-6adf60d7a6b2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27582025-04-16T04:02:55.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2054 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dfe40c9c-7629-410d-bcf6-6adf60d7a6b2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27592025-04-16T04:02:56.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c467b57-0a85-45ff-ae30-b9908c1d875b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27602025-04-16T04:02:56.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c467b57-0a85-45ff-ae30-b9908c1d875b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27612025-04-16T04:02:56.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2148 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c467b57-0a85-45ff-ae30-b9908c1d875b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27622025-04-16T04:02:57.573ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61daab5e-652d-405b-ba20-36fed5aaddaf roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27632025-04-16T04:02:57.573ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61daab5e-652d-405b-ba20-36fed5aaddaf resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27642025-04-16T04:02:57.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2211 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 61daab5e-652d-405b-ba20-36fed5aaddaf response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27652025-04-16T04:02:58.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 511 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = d6b24a8c-f49d-48a1-844a-d2cb34b0ef5a response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
27662025-04-16T04:02:58.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f44ff74-0829-44f5-983b-f7ce2aee6c02 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27672025-04-16T04:02:58.594ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f44ff74-0829-44f5-983b-f7ce2aee6c02 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27682025-04-16T04:02:58.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2008 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f44ff74-0829-44f5-983b-f7ce2aee6c02 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27692025-04-16T04:02:59.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
27702025-04-16T04:02:59.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
27712025-04-16T04:02:59.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27722025-04-16T04:02:59.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27732025-04-16T04:02:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27742025-04-16T04:02:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27752025-04-16T04:02:59.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27762025-04-16T04:02:59.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27772025-04-16T04:02:59.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27782025-04-16T04:02:59.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.56806ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27792025-04-16T04:02:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27802025-04-16T04:02:59.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27812025-04-16T04:02:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27822025-04-16T04:02:59.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27832025-04-16T04:02:59.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27842025-04-16T04:02:59.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27852025-04-16T04:02:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
27862025-04-16T04:02:59.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27872025-04-16T04:02:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27882025-04-16T04:02:59.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
27892025-04-16T04:02:59.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.148028ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
27902025-04-16T04:02:59.618ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb115bcd-a8e6-446d-a517-65c79d58788a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27912025-04-16T04:02:59.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb115bcd-a8e6-446d-a517-65c79d58788a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27922025-04-16T04:02:59.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1585 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb115bcd-a8e6-446d-a517-65c79d58788a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27932025-04-16T04:03:00.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc0742db-092c-40dc-b791-7ace0d146048 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27942025-04-16T04:03:00.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc0742db-092c-40dc-b791-7ace0d146048 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27952025-04-16T04:03:00.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2131 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc0742db-092c-40dc-b791-7ace0d146048 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27962025-04-16T04:03:01.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c5fc74e8-7a64-45af-81d1-b87f806545c3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27972025-04-16T04:03:01.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c5fc74e8-7a64-45af-81d1-b87f806545c3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27982025-04-16T04:03:01.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c5fc74e8-7a64-45af-81d1-b87f806545c3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
27992025-04-16T04:03:02.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac469292-14f8-47c6-be4d-ab4d955dd83b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28002025-04-16T04:03:02.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac469292-14f8-47c6-be4d-ab4d955dd83b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28012025-04-16T04:03:02.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2110 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac469292-14f8-47c6-be4d-ab4d955dd83b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28022025-04-16T04:03:03.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3b2c411f-63a2-475b-b8a1-c94e1f39e31b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28032025-04-16T04:03:03.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3b2c411f-63a2-475b-b8a1-c94e1f39e31b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28042025-04-16T04:03:03.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3b2c411f-63a2-475b-b8a1-c94e1f39e31b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28052025-04-16T04:03:04.728ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d96c80d-112f-4d4e-aa4b-504448fb06de roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28062025-04-16T04:03:04.729ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d96c80d-112f-4d4e-aa4b-504448fb06de resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28072025-04-16T04:03:04.730ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d96c80d-112f-4d4e-aa4b-504448fb06de response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28082025-04-16T04:03:05.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3db77ec4-9df3-485e-aba3-d64eb5145651 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28092025-04-16T04:03:05.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3db77ec4-9df3-485e-aba3-d64eb5145651 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28102025-04-16T04:03:05.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2419 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3db77ec4-9df3-485e-aba3-d64eb5145651 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28112025-04-16T04:03:06.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d7c4a240-f904-4036-b329-9e877cf22792 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28122025-04-16T04:03:06.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d7c4a240-f904-4036-b329-9e877cf22792 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28132025-04-16T04:03:06.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1829 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d7c4a240-f904-4036-b329-9e877cf22792 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28142025-04-16T04:03:07.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ff33987-e6f6-4283-9ce2-ceb95a6441f3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28152025-04-16T04:03:07.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ff33987-e6f6-4283-9ce2-ceb95a6441f3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28162025-04-16T04:03:07.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2042 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ff33987-e6f6-4283-9ce2-ceb95a6441f3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28172025-04-16T04:03:08.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 759 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 9d4da37c-4268-461c-9c17-1d41c80e12f3 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
28182025-04-16T04:03:08.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0ee02dc-f344-4b4b-9d98-8ca1a9698774 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28192025-04-16T04:03:08.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0ee02dc-f344-4b4b-9d98-8ca1a9698774 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28202025-04-16T04:03:08.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3061 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0ee02dc-f344-4b4b-9d98-8ca1a9698774 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28212025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
28222025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
28232025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28242025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28252025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28262025-04-16T04:03:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28272025-04-16T04:03:09.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28282025-04-16T04:03:09.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28292025-04-16T04:03:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28302025-04-16T04:03:09.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.385937ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28312025-04-16T04:03:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28322025-04-16T04:03:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28332025-04-16T04:03:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28342025-04-16T04:03:09.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28352025-04-16T04:03:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28362025-04-16T04:03:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28372025-04-16T04:03:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
28382025-04-16T04:03:09.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28392025-04-16T04:03:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28402025-04-16T04:03:09.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
28412025-04-16T04:03:09.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.859592ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
28422025-04-16T04:03:09.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f35729c6-18b2-423a-9aa5-5e3b6a4cb168 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28432025-04-16T04:03:09.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f35729c6-18b2-423a-9aa5-5e3b6a4cb168 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28442025-04-16T04:03:09.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2198 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f35729c6-18b2-423a-9aa5-5e3b6a4cb168 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28452025-04-16T04:03:10.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 291a22db-349b-4872-a832-ae46d673a70c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28462025-04-16T04:03:10.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 291a22db-349b-4872-a832-ae46d673a70c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28472025-04-16T04:03:10.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2311 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 291a22db-349b-4872-a832-ae46d673a70c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28482025-04-16T04:03:11.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afb52b04-2105-4c2d-8579-8ee23255a033 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28492025-04-16T04:03:11.889ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afb52b04-2105-4c2d-8579-8ee23255a033 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28502025-04-16T04:03:11.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2188 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afb52b04-2105-4c2d-8579-8ee23255a033 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28512025-04-16T04:03:12.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9320c57c-1f47-4fd6-b816-2bb80a8ba08f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28522025-04-16T04:03:12.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9320c57c-1f47-4fd6-b816-2bb80a8ba08f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28532025-04-16T04:03:12.914ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3486 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9320c57c-1f47-4fd6-b816-2bb80a8ba08f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28542025-04-16T04:03:13.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 4c5163bc-1acd-4b91-96f1-f87bccf90dbf roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
28552025-04-16T04:03:13.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 4c5163bc-1acd-4b91-96f1-f87bccf90dbf resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
28562025-04-16T04:03:13.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2918 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 4c5163bc-1acd-4b91-96f1-f87bccf90dbf response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
28572025-04-16T04:03:13.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 721dcf86-6dd0-4094-bee2-34019c35044a roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
28582025-04-16T04:03:13.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 721dcf86-6dd0-4094-bee2-34019c35044a resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
28592025-04-16T04:03:13.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2646 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 721dcf86-6dd0-4094-bee2-34019c35044a response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
28602025-04-16T04:03:13.940ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 330bd462-c24e-465e-bd47-9775989a70c8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28612025-04-16T04:03:13.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 330bd462-c24e-465e-bd47-9775989a70c8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28622025-04-16T04:03:13.942ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2073 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 330bd462-c24e-465e-bd47-9775989a70c8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28632025-04-16T04:03:14.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65463cde-ec45-4eda-a958-7f61de644781 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28642025-04-16T04:03:14.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65463cde-ec45-4eda-a958-7f61de644781 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28652025-04-16T04:03:14.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2117 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65463cde-ec45-4eda-a958-7f61de644781 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28662025-04-16T04:03:15.987ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 516ba7e0-4fbd-4c44-ba0f-e45c35d3b432 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28672025-04-16T04:03:15.988ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 516ba7e0-4fbd-4c44-ba0f-e45c35d3b432 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28682025-04-16T04:03:15.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2527 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 516ba7e0-4fbd-4c44-ba0f-e45c35d3b432 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28692025-04-16T04:03:17.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b28d8a9a-d4c2-4b14-960b-76326e707c3c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28702025-04-16T04:03:17.011ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b28d8a9a-d4c2-4b14-960b-76326e707c3c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28712025-04-16T04:03:17.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2074 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b28d8a9a-d4c2-4b14-960b-76326e707c3c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28722025-04-16T04:03:18.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b3e6b4d5-d453-4db7-87d1-9154df79eb7a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28732025-04-16T04:03:18.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b3e6b4d5-d453-4db7-87d1-9154df79eb7a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28742025-04-16T04:03:18.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2147 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b3e6b4d5-d453-4db7-87d1-9154df79eb7a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28752025-04-16T04:03:18.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 484 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 0ea7d712-2423-4ad5-93a7-6f727fb8c074 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
28762025-04-16T04:03:19.053ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 988ffe65-4886-4f63-ad43-7cc52e2f5ce6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28772025-04-16T04:03:19.053ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 988ffe65-4886-4f63-ad43-7cc52e2f5ce6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28782025-04-16T04:03:19.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2134 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 988ffe65-4886-4f63-ad43-7cc52e2f5ce6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
28792025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
28802025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
28812025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28822025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
28832025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
28842025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28852025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28862025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
28872025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28882025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
28892025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
28902025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
28912025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
28922025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
28932025-04-16T04:03:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28942025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
28952025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28962025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28972025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
28982025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
28992025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29002025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
29012025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29022025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
29032025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
29042025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
29052025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29062025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
29072025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
29082025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
29092025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
29102025-04-16T04:03:19.446ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
29112025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29122025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
29132025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.54\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
29142025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
29152025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29162025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29172025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
29182025-04-16T04:03:19.446ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
29192025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
29202025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
29212025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
29222025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
29232025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
29242025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
29252025-04-16T04:03:19.446ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:53:19.446628570 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
29262025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
29272025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
29282025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29292025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
29302025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
29312025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29322025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
29332025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29342025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
29352025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29362025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
29372025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29382025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
29392025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29402025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
29412025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
29422025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
29432025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
29442025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
29452025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29462025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29472025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29482025-04-16T04:03:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29492025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29502025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29512025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29522025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
29532025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29542025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
29552025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
29562025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
29572025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29582025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29592025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
29602025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29612025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29622025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
29632025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29642025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
29652025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29662025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
29672025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29682025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29692025-04-16T04:03:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3089, tv_nsec: 728111297 } })
29702025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
29712025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29722025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29732025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29742025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29752025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29762025-04-16T04:03:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3089, tv_nsec: 728254060 } })
29772025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
29782025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29792025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
29802025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29812025-04-16T04:03:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
29822025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
29832025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
29842025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.51171ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
29852025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
29862025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.582211ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
29872025-04-16T04:03:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
29882025-04-16T04:03:19.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29892025-04-16T04:03:19.450ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.04942ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
29902025-04-16T04:03:19.450ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3089, tv_nsec: 731662757 } })
29912025-04-16T04:03:19.451ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
29922025-04-16T04:03:19.452ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932025-04-16T04:03:19.452ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29942025-04-16T04:03:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29952025-04-16T04:03:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29962025-04-16T04:03:19.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29972025-04-16T04:03:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29982025-04-16T04:03:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29992025-04-16T04:03:19.454ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30002025-04-16T04:03:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30012025-04-16T04:03:19.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30022025-04-16T04:03:19.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.744263ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
30032025-04-16T04:03:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30042025-04-16T04:03:19.455ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30052025-04-16T04:03:19.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30062025-04-16T04:03:19.456ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.920746ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
30072025-04-16T04:03:19.456ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30082025-04-16T04:03:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30092025-04-16T04:03:19.456ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.300854ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
30102025-04-16T04:03:19.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
30112025-04-16T04:03:19.458ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30122025-04-16T04:03:19.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
30132025-04-16T04:03:19.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30142025-04-16T04:03:19.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30152025-04-16T04:03:19.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30162025-04-16T04:03:19.459ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30172025-04-16T04:03:19.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30182025-04-16T04:03:19.460ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30192025-04-16T04:03:19.460ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3089, tv_nsec: 741409960 } })
30202025-04-16T04:03:19.460ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
30212025-04-16T04:03:19.461ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30222025-04-16T04:03:19.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30232025-04-16T04:03:19.461ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:53:19.446628570 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
30242025-04-16T04:03:19.461ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.875794ms iteration = 10 status = {"expiration":"2025-04-16T03:53:19.446628570Z","pruned":[]}
30252025-04-16T04:03:19.461ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30262025-04-16T04:03:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
30272025-04-16T04:03:19.462ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30282025-04-16T04:03:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30292025-04-16T04:03:19.464ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 17.531277ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
30302025-04-16T04:03:19.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30312025-04-16T04:03:19.464ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30322025-04-16T04:03:19.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30332025-04-16T04:03:19.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30342025-04-16T04:03:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30352025-04-16T04:03:19.469ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30362025-04-16T04:03:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30372025-04-16T04:03:19.471ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30382025-04-16T04:03:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
30392025-04-16T04:03:19.473ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30402025-04-16T04:03:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30412025-04-16T04:03:19.481ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3089, tv_nsec: 762124478 } })
30422025-04-16T04:03:19.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432025-04-16T04:03:19.481ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
30442025-04-16T04:03:19.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30452025-04-16T04:03:19.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30462025-04-16T04:03:19.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30472025-04-16T04:03:19.493ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
30482025-04-16T04:03:19.493ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
30492025-04-16T04:03:19.493ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
30502025-04-16T04:03:19.493ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30512025-04-16T04:03:19.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30522025-04-16T04:03:19.497ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30532025-04-16T04:03:19.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30542025-04-16T04:03:19.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30552025-04-16T04:03:19.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30562025-04-16T04:03:19.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30572025-04-16T04:03:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30582025-04-16T04:03:19.510ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
30592025-04-16T04:03:19.510ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
30602025-04-16T04:03:19.510ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.530003ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
30612025-04-16T04:03:19.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30622025-04-16T04:03:19.511ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30632025-04-16T04:03:19.511ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30642025-04-16T04:03:19.512ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30652025-04-16T04:03:19.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30662025-04-16T04:03:19.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30672025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30682025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30692025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
30702025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30712025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
30722025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30732025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30742025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30752025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30762025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30772025-04-16T04:03:19.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30782025-04-16T04:03:19.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30792025-04-16T04:03:19.518ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30802025-04-16T04:03:19.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30812025-04-16T04:03:19.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30822025-04-16T04:03:19.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30832025-04-16T04:03:19.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30842025-04-16T04:03:19.521ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30852025-04-16T04:03:19.521ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
30862025-04-16T04:03:19.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30872025-04-16T04:03:19.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30882025-04-16T04:03:19.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30892025-04-16T04:03:19.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30902025-04-16T04:03:19.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30912025-04-16T04:03:19.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30922025-04-16T04:03:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30932025-04-16T04:03:19.523ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30942025-04-16T04:03:19.524ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30952025-04-16T04:03:19.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 78.749714ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30962025-04-16T04:03:19.525ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30972025-04-16T04:03:19.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30982025-04-16T04:03:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30992025-04-16T04:03:19.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31002025-04-16T04:03:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31012025-04-16T04:03:19.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31022025-04-16T04:03:19.527ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31032025-04-16T04:03:19.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31042025-04-16T04:03:19.527ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
31052025-04-16T04:03:19.528ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31062025-04-16T04:03:19.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
31072025-04-16T04:03:19.529ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31082025-04-16T04:03:19.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31092025-04-16T04:03:19.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31102025-04-16T04:03:19.529ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
31112025-04-16T04:03:19.529ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
31122025-04-16T04:03:19.529ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 83.232933ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
31132025-04-16T04:03:19.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31142025-04-16T04:03:19.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31152025-04-16T04:03:19.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31162025-04-16T04:03:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31172025-04-16T04:03:19.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31182025-04-16T04:03:19.533ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31192025-04-16T04:03:19.533ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.761113ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31202025-04-16T04:03:19.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
31212025-04-16T04:03:19.533ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 87.12885ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
31222025-04-16T04:03:19.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31232025-04-16T04:03:19.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31242025-04-16T04:03:19.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31252025-04-16T04:03:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31262025-04-16T04:03:19.542ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31272025-04-16T04:03:19.542ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
31282025-04-16T04:03:19.542ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31292025-04-16T04:03:19.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31302025-04-16T04:03:19.543ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 96.609197ms iteration = 5 status = {"nsuccess":5}
31312025-04-16T04:03:19.543ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31322025-04-16T04:03:19.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31332025-04-16T04:03:19.547ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31342025-04-16T04:03:19.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31352025-04-16T04:03:19.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31362025-04-16T04:03:19.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31372025-04-16T04:03:19.561ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31382025-04-16T04:03:19.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31392025-04-16T04:03:19.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31402025-04-16T04:03:19.564ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31412025-04-16T04:03:19.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31422025-04-16T04:03:19.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
31432025-04-16T04:03:19.567ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31442025-04-16T04:03:19.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31452025-04-16T04:03:19.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 123.09681ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
31462025-04-16T04:03:19.570ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 124.09761ms iteration = 19 status = 0
31472025-04-16T04:03:19.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 124.929226ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31482025-04-16T04:03:19.571ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
31492025-04-16T04:03:19.571ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 124.765183ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
31502025-04-16T04:03:19.576ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31512025-04-16T04:03:19.576ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
31522025-04-16T04:03:19.576ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31532025-04-16T04:03:19.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31542025-04-16T04:03:19.578ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31552025-04-16T04:03:19.580ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31562025-04-16T04:03:19.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31572025-04-16T04:03:19.582ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31582025-04-16T04:03:19.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31592025-04-16T04:03:19.585ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
31602025-04-16T04:03:19.585ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31612025-04-16T04:03:19.585ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
31622025-04-16T04:03:19.585ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 138.691307ms iteration = 10 status = {"generation":2}
31632025-04-16T04:03:19.591ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31642025-04-16T04:03:19.591ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31652025-04-16T04:03:19.591ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31662025-04-16T04:03:19.591ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31672025-04-16T04:03:19.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31682025-04-16T04:03:19.593ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31692025-04-16T04:03:19.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31702025-04-16T04:03:19.598ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31712025-04-16T04:03:19.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31722025-04-16T04:03:19.600ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
31732025-04-16T04:03:19.600ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31742025-04-16T04:03:19.600ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b23f320-d6a7-4c6f-82e3-6fe3ca16ee2d", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
31752025-04-16T04:03:19.600ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31762025-04-16T04:03:19.603ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31772025-04-16T04:03:19.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31782025-04-16T04:03:19.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31792025-04-16T04:03:19.607ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31802025-04-16T04:03:19.607ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
31812025-04-16T04:03:19.607ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 160.979357ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
31822025-04-16T04:03:19.616ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
31832025-04-16T04:03:19.616ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 169.434164ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
31842025-04-16T04:03:19.631ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
31852025-04-16T04:03:19.631ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31862025-04-16T04:03:19.632ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8588fb4d-9528-4d0d-a39c-4c3a1b03828f", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
31872025-04-16T04:03:19.632ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31882025-04-16T04:03:19.634ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
31892025-04-16T04:03:19.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31902025-04-16T04:03:19.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31912025-04-16T04:03:19.649ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-04-16T04:03:19.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31932025-04-16T04:03:19.651ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31942025-04-16T04:03:19.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31952025-04-16T04:03:19.657ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
31962025-04-16T04:03:19.657ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 210.691219ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
31972025-04-16T04:03:19.660ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
31982025-04-16T04:03:19.660ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31992025-04-16T04:03:19.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32002025-04-16T04:03:19.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32012025-04-16T04:03:19.663ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32022025-04-16T04:03:19.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32032025-04-16T04:03:19.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32042025-04-16T04:03:19.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32052025-04-16T04:03:19.699ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 252.594086ms iteration = 21 status = {"physical_disks_added":0}
32062025-04-16T04:03:19.736ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
32072025-04-16T04:03:19.777ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
32082025-04-16T04:03:19.777ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32092025-04-16T04:03:19.777ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d5e7e2e-42ed-4512-9264-d211690dd359", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32102025-04-16T04:03:19.777ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32112025-04-16T04:03:19.778ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
32122025-04-16T04:03:19.779ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32132025-04-16T04:03:19.791ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 344.962219ms iteration = 19 status = {}
32142025-04-16T04:03:19.857ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32152025-04-16T04:03:19.857ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32162025-04-16T04:03:19.857ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "728397fe-3656-4150-bffb-1d7e58f71731", "content-length": "372", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32172025-04-16T04:03:19.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32182025-04-16T04:03:19.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32192025-04-16T04:03:19.858ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32202025-04-16T04:03:19.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32212025-04-16T04:03:19.862ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32222025-04-16T04:03:19.862ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32232025-04-16T04:03:19.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32242025-04-16T04:03:19.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32252025-04-16T04:03:19.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32262025-04-16T04:03:19.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32272025-04-16T04:03:19.867ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
32282025-04-16T04:03:19.867ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
32292025-04-16T04:03:19.867ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
32302025-04-16T04:03:19.867ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 421.256815ms iteration = 10 status = {"generation":1}
32312025-04-16T04:03:19.885ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32322025-04-16T04:03:19.885ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32332025-04-16T04:03:19.885ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "53d04e6f-4207-4ed7-8b5e-b34641ce002d", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32342025-04-16T04:03:19.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32352025-04-16T04:03:19.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32362025-04-16T04:03:19.889ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32372025-04-16T04:03:19.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32382025-04-16T04:03:19.894ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32392025-04-16T04:03:19.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32402025-04-16T04:03:19.898ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32412025-04-16T04:03:19.898ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32422025-04-16T04:03:19.899ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5144437f-989f-4cca-b097-59584a27ae5f", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32432025-04-16T04:03:19.901ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 454.674054ms iteration = 19 status = {}
32442025-04-16T04:03:19.911ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
32452025-04-16T04:03:19.912ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 465.12857ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
32462025-04-16T04:03:19.938ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
32472025-04-16T04:03:19.939ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "d0300138-896f-4eee-8bca-bdc0975cad7c", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32482025-04-16T04:03:19.939ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32492025-04-16T04:03:19.940ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4d455c40-5837-45a6-9823-9027b2ca4cfe", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32502025-04-16T04:03:19.940ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32512025-04-16T04:03:19.942ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f7335326-bb1a-450d-bef1-f698cf58c4ce", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32522025-04-16T04:03:19.942ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32532025-04-16T04:03:19.944ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
32542025-04-16T04:03:19.944ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 498.634172ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
32552025-04-16T04:03:19.955ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32562025-04-16T04:03:19.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32572025-04-16T04:03:19.958ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 512.453734ms iteration = 19 status = null
32582025-04-16T04:03:19.996ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
32592025-04-16T04:03:19.997ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a774eaa3-b7cb-4c0f-8a0b-a1cf571e7e30", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32602025-04-16T04:03:19.997ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32612025-04-16T04:03:19.999ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "6a776539-0128-40e2-8889-921baf2fc40a", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32622025-04-16T04:03:19.999ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32632025-04-16T04:03:20.001ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32642025-04-16T04:03:20.001ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 554.495704ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
32652025-04-16T04:03:20.054ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32662025-04-16T04:03:20.054ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "29bf9b39-0533-4de3-9a93-7ba9adee90f1", "content-length": "691", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32672025-04-16T04:03:20.054ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32682025-04-16T04:03:20.054ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32692025-04-16T04:03:20.054ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32702025-04-16T04:03:20.057ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 610.715354ms iteration = 19 status = {}
32712025-04-16T04:03:20.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32722025-04-16T04:03:20.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32732025-04-16T04:03:20.074ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aa938680-2015-4a5f-b4fc-91c88d774ce5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
32742025-04-16T04:03:20.074ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aa938680-2015-4a5f-b4fc-91c88d774ce5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
32752025-04-16T04:03:20.075ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
32762025-04-16T04:03:20.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32772025-04-16T04:03:20.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32782025-04-16T04:03:20.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2026 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aa938680-2015-4a5f-b4fc-91c88d774ce5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
32792025-04-16T04:03:20.087ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
32802025-04-16T04:03:20.087ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
32812025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
32822025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
32832025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32842025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32852025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32862025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32872025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32882025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32892025-04-16T04:03:20.088ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
32902025-04-16T04:03:20.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32912025-04-16T04:03:20.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32922025-04-16T04:03:20.091ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
32932025-04-16T04:03:20.091ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32942025-04-16T04:03:20.197ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "80525eef-c79b-47bc-806d-772877d6f4c2", "date": "Wed, 16 Apr 2025 04:03:19 GMT"} })
32952025-04-16T04:03:20.197ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
32962025-04-16T04:03:20.197ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32972025-04-16T04:03:20.197ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e10973e-fd74-4c60-88bf-80a4d74e5079", "content-length": "682", "date": "Wed, 16 Apr 2025 04:03:20 GMT"} })
32982025-04-16T04:03:20.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32992025-04-16T04:03:20.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33002025-04-16T04:03:20.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33012025-04-16T04:03:20.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33022025-04-16T04:03:20.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33032025-04-16T04:03:20.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33042025-04-16T04:03:20.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33052025-04-16T04:03:20.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33062025-04-16T04:03:20.216ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 769.864415ms iteration = 19 status = {}
33072025-04-16T04:03:21.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aeb9bd7d-4214-4794-a3bc-30a9db1034a8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33082025-04-16T04:03:21.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aeb9bd7d-4214-4794-a3bc-30a9db1034a8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33092025-04-16T04:03:21.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2515 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aeb9bd7d-4214-4794-a3bc-30a9db1034a8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33102025-04-16T04:03:22.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42df1dc7-6b16-4188-9b33-45bd3924fa9b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33112025-04-16T04:03:22.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42df1dc7-6b16-4188-9b33-45bd3924fa9b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33122025-04-16T04:03:22.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1985 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42df1dc7-6b16-4188-9b33-45bd3924fa9b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33132025-04-16T04:03:23.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b9927bac-f301-4e58-84d5-8b4fe0ba2078 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33142025-04-16T04:03:23.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b9927bac-f301-4e58-84d5-8b4fe0ba2078 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33152025-04-16T04:03:23.244ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b9927bac-f301-4e58-84d5-8b4fe0ba2078 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33162025-04-16T04:03:24.265ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4904af22-dd5e-48e9-b95a-0244ec74678e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33172025-04-16T04:03:24.265ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4904af22-dd5e-48e9-b95a-0244ec74678e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33182025-04-16T04:03:24.267ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1777 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4904af22-dd5e-48e9-b95a-0244ec74678e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33192025-04-16T04:03:25.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0a7a66e-b41e-480f-9a9b-5d7bd8582b91 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33202025-04-16T04:03:25.288ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0a7a66e-b41e-480f-9a9b-5d7bd8582b91 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33212025-04-16T04:03:25.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2947 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0a7a66e-b41e-480f-9a9b-5d7bd8582b91 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33222025-04-16T04:03:26.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e9469f44-d637-4386-9d27-6ca0d74002c3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33232025-04-16T04:03:26.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e9469f44-d637-4386-9d27-6ca0d74002c3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33242025-04-16T04:03:26.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2151 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e9469f44-d637-4386-9d27-6ca0d74002c3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33252025-04-16T04:03:27.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3446e440-c614-41c5-a2b2-181de1b8ebea roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33262025-04-16T04:03:27.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3446e440-c614-41c5-a2b2-181de1b8ebea resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33272025-04-16T04:03:27.333ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3446e440-c614-41c5-a2b2-181de1b8ebea response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33282025-04-16T04:03:28.353ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39335c73-fe48-4c92-a7f1-09e486f07e15 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33292025-04-16T04:03:28.353ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39335c73-fe48-4c92-a7f1-09e486f07e15 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33302025-04-16T04:03:28.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2295 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39335c73-fe48-4c92-a7f1-09e486f07e15 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33312025-04-16T04:03:28.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 479 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = bd0b4715-2155-4ec5-b26b-796662c378af response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
33322025-04-16T04:03:29.376ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e0d0722-f621-4b18-9edf-63fc87be2ff8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33332025-04-16T04:03:29.376ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e0d0722-f621-4b18-9edf-63fc87be2ff8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33342025-04-16T04:03:29.377ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e0d0722-f621-4b18-9edf-63fc87be2ff8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33352025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
33362025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33372025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
33382025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33392025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33402025-04-16T04:03:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33412025-04-16T04:03:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33422025-04-16T04:03:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33432025-04-16T04:03:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33442025-04-16T04:03:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33452025-04-16T04:03:29.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33462025-04-16T04:03:29.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33472025-04-16T04:03:29.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33482025-04-16T04:03:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
33492025-04-16T04:03:29.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33502025-04-16T04:03:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33512025-04-16T04:03:29.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
33522025-04-16T04:03:29.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.562266ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
33532025-04-16T04:03:29.476ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33542025-04-16T04:03:29.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33552025-04-16T04:03:29.478ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.748796ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33562025-04-16T04:03:30.396ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4bc9d865-b840-4a7b-83b0-0af1dbd54a87 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33572025-04-16T04:03:30.396ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4bc9d865-b840-4a7b-83b0-0af1dbd54a87 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33582025-04-16T04:03:30.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2033 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4bc9d865-b840-4a7b-83b0-0af1dbd54a87 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33592025-04-16T04:03:31.418ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aaccf728-7ce8-4f50-9150-cf13d0ae430f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33602025-04-16T04:03:31.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aaccf728-7ce8-4f50-9150-cf13d0ae430f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33612025-04-16T04:03:31.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2096 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = aaccf728-7ce8-4f50-9150-cf13d0ae430f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33622025-04-16T04:03:32.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f78d723d-a0a8-4b22-ad0b-ffd591e1ffca roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33632025-04-16T04:03:32.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f78d723d-a0a8-4b22-ad0b-ffd591e1ffca resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33642025-04-16T04:03:32.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f78d723d-a0a8-4b22-ad0b-ffd591e1ffca response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33652025-04-16T04:03:33.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b2ab0771-7c08-4e7a-bb59-e9700c6dc00f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33662025-04-16T04:03:33.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b2ab0771-7c08-4e7a-bb59-e9700c6dc00f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33672025-04-16T04:03:33.463ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1994 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b2ab0771-7c08-4e7a-bb59-e9700c6dc00f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33682025-04-16T04:03:34.482ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffdfa47b-550d-4ad7-b1dd-7ade46b90051 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33692025-04-16T04:03:34.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffdfa47b-550d-4ad7-b1dd-7ade46b90051 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33702025-04-16T04:03:34.484ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2354 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffdfa47b-550d-4ad7-b1dd-7ade46b90051 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33712025-04-16T04:03:35.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 484f62b5-db54-4815-8814-207b5f744339 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33722025-04-16T04:03:35.504ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 484f62b5-db54-4815-8814-207b5f744339 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33732025-04-16T04:03:35.506ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 484f62b5-db54-4815-8814-207b5f744339 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33742025-04-16T04:03:36.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76da10e4-c199-4b1b-bf24-d86e8c3170b4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33752025-04-16T04:03:36.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76da10e4-c199-4b1b-bf24-d86e8c3170b4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33762025-04-16T04:03:36.528ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2537 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 76da10e4-c199-4b1b-bf24-d86e8c3170b4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33772025-04-16T04:03:37.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b87ccc5a-9ea3-403b-a371-9d30c633997a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33782025-04-16T04:03:37.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b87ccc5a-9ea3-403b-a371-9d30c633997a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33792025-04-16T04:03:37.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2028 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b87ccc5a-9ea3-403b-a371-9d30c633997a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33802025-04-16T04:03:38.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 485 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 12f461b6-8729-4be8-8889-a223df4a2f11 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
33812025-04-16T04:03:38.571ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77e1fe60-fba4-461d-9ee7-e0c3cf654756 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33822025-04-16T04:03:38.571ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77e1fe60-fba4-461d-9ee7-e0c3cf654756 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33832025-04-16T04:03:38.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1755 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77e1fe60-fba4-461d-9ee7-e0c3cf654756 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
33842025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
33852025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
33862025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33872025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33882025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33892025-04-16T04:03:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33902025-04-16T04:03:39.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33912025-04-16T04:03:39.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33922025-04-16T04:03:39.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33932025-04-16T04:03:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33942025-04-16T04:03:39.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33952025-04-16T04:03:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33962025-04-16T04:03:39.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33972025-04-16T04:03:39.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33982025-04-16T04:03:39.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33992025-04-16T04:03:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
34002025-04-16T04:03:39.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34012025-04-16T04:03:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34022025-04-16T04:03:39.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
34032025-04-16T04:03:39.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.216759ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
34042025-04-16T04:03:39.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.351229ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34052025-04-16T04:03:39.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f59a2d0f-db1d-437e-a85d-555d39496293 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34062025-04-16T04:03:39.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f59a2d0f-db1d-437e-a85d-555d39496293 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34072025-04-16T04:03:39.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1871 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f59a2d0f-db1d-437e-a85d-555d39496293 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34082025-04-16T04:03:40.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97a727dd-d1cc-4f92-9b47-ae53fdc31b6b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34092025-04-16T04:03:40.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97a727dd-d1cc-4f92-9b47-ae53fdc31b6b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34102025-04-16T04:03:40.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2218 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 97a727dd-d1cc-4f92-9b47-ae53fdc31b6b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34112025-04-16T04:03:41.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a883cbbd-d527-43e0-b750-ef9acc50b157 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34122025-04-16T04:03:41.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a883cbbd-d527-43e0-b750-ef9acc50b157 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34132025-04-16T04:03:41.640ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2132 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = a883cbbd-d527-43e0-b750-ef9acc50b157 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34142025-04-16T04:03:42.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cdc6052f-1360-4024-9a3c-91f305e907cf roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34152025-04-16T04:03:42.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cdc6052f-1360-4024-9a3c-91f305e907cf resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34162025-04-16T04:03:42.661ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1961 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cdc6052f-1360-4024-9a3c-91f305e907cf response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34172025-04-16T04:03:43.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 6ff756bc-15c4-43d1-abff-a261ce52e9e9 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
34182025-04-16T04:03:43.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 6ff756bc-15c4-43d1-abff-a261ce52e9e9 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
34192025-04-16T04:03:43.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3472 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 6ff756bc-15c4-43d1-abff-a261ce52e9e9 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
34202025-04-16T04:03:43.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 271c34b5-8085-4b94-88ed-2059850c7a66 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
34212025-04-16T04:03:43.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 271c34b5-8085-4b94-88ed-2059850c7a66 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
34222025-04-16T04:03:43.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58028 req_id = 271c34b5-8085-4b94-88ed-2059850c7a66 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
34232025-04-16T04:03:43.681ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d14c40ba-4fba-48af-b5e0-31adcb35048e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34242025-04-16T04:03:43.681ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d14c40ba-4fba-48af-b5e0-31adcb35048e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34252025-04-16T04:03:43.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d14c40ba-4fba-48af-b5e0-31adcb35048e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34262025-04-16T04:03:44.703ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f82da03-bc58-4917-b077-406a5d9035bd roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34272025-04-16T04:03:44.703ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f82da03-bc58-4917-b077-406a5d9035bd resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34282025-04-16T04:03:44.705ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2285 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f82da03-bc58-4917-b077-406a5d9035bd response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34292025-04-16T04:03:45.724ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f77e3aaf-a0b8-4bdf-b8b4-88729db390d5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34302025-04-16T04:03:45.724ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f77e3aaf-a0b8-4bdf-b8b4-88729db390d5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34312025-04-16T04:03:45.726ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1579 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f77e3aaf-a0b8-4bdf-b8b4-88729db390d5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34322025-04-16T04:03:46.745ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 01e867ef-1c1b-457b-a579-eb79e2ff5569 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34332025-04-16T04:03:46.745ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 01e867ef-1c1b-457b-a579-eb79e2ff5569 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34342025-04-16T04:03:46.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2070 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 01e867ef-1c1b-457b-a579-eb79e2ff5569 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34352025-04-16T04:03:47.766ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03d3f622-9ee1-42fc-b416-c6335e3741b4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34362025-04-16T04:03:47.767ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03d3f622-9ee1-42fc-b416-c6335e3741b4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34372025-04-16T04:03:47.769ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2403 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 03d3f622-9ee1-42fc-b416-c6335e3741b4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34382025-04-16T04:03:48.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 532 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = bfa5107f-2c7e-4029-8f63-28a10c6f4e8a response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
34392025-04-16T04:03:48.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5eb53f9d-b8e3-4648-916f-aefb682ae2b5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34402025-04-16T04:03:48.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5eb53f9d-b8e3-4648-916f-aefb682ae2b5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34412025-04-16T04:03:48.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2201 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5eb53f9d-b8e3-4648-916f-aefb682ae2b5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
34422025-04-16T04:03:49.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
34432025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
34442025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34452025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34462025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
34472025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
34482025-04-16T04:03:49.447ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
34492025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
34502025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34512025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34522025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
34532025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
34542025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
34552025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
34562025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34572025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
34582025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34592025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
34602025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34612025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
34622025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34632025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
34642025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34652025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
34662025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
34672025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
34682025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34692025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
34702025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
34712025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
34722025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34732025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
34742025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
34752025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
34762025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
34772025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34782025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34792025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
34802025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34812025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34822025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
34832025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34842025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34852025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34862025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34872025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34882025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34892025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34902025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34912025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
34922025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34932025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34942025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34952025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34962025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
34972025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
34982025-04-16T04:03:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34992025-04-16T04:03:49.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35002025-04-16T04:03:49.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.770404ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
35012025-04-16T04:03:49.450ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35022025-04-16T04:03:49.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35032025-04-16T04:03:49.450ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.399857ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
35042025-04-16T04:03:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35052025-04-16T04:03:49.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35062025-04-16T04:03:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35072025-04-16T04:03:49.453ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35082025-04-16T04:03:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35092025-04-16T04:03:49.454ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35102025-04-16T04:03:49.454ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35112025-04-16T04:03:49.456ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3119, tv_nsec: 737168746 } })
35122025-04-16T04:03:49.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
35132025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
35142025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35152025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35162025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
35172025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35182025-04-16T04:03:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35192025-04-16T04:03:49.459ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35202025-04-16T04:03:49.459ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35212025-04-16T04:03:49.461ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3119, tv_nsec: 741983091 } })
35222025-04-16T04:03:49.461ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
35232025-04-16T04:03:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35242025-04-16T04:03:49.461ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35252025-04-16T04:03:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35262025-04-16T04:03:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35272025-04-16T04:03:49.462ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35282025-04-16T04:03:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35292025-04-16T04:03:49.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35302025-04-16T04:03:49.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35312025-04-16T04:03:49.466ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3119, tv_nsec: 746745245 } })
35322025-04-16T04:03:49.466ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
35332025-04-16T04:03:49.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35342025-04-16T04:03:49.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35352025-04-16T04:03:49.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35362025-04-16T04:03:49.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
35372025-04-16T04:03:49.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35382025-04-16T04:03:49.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35392025-04-16T04:03:49.489ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
35402025-04-16T04:03:49.489ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.415477ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
35412025-04-16T04:03:49.489ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35422025-04-16T04:03:49.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35432025-04-16T04:03:49.495ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35442025-04-16T04:03:49.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35452025-04-16T04:03:49.498ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35462025-04-16T04:03:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35472025-04-16T04:03:49.500ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35482025-04-16T04:03:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35492025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 54.225141ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
35502025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35512025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35522025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35532025-04-16T04:03:49.501ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35542025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
35552025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35562025-04-16T04:03:49.501ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35572025-04-16T04:03:49.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35582025-04-16T04:03:49.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35592025-04-16T04:03:49.503ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 55.924524ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35602025-04-16T04:03:49.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35612025-04-16T04:03:49.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35622025-04-16T04:03:49.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35632025-04-16T04:03:49.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35642025-04-16T04:03:49.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35652025-04-16T04:03:49.506ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 59.526115ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
35662025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35672025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35682025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35692025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35702025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35712025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35722025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35732025-04-16T04:03:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35742025-04-16T04:03:49.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35752025-04-16T04:03:49.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.612346ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35762025-04-16T04:03:49.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 64.554454ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35772025-04-16T04:03:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35782025-04-16T04:03:49.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35792025-04-16T04:03:49.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
35802025-04-16T04:03:49.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35812025-04-16T04:03:49.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35822025-04-16T04:03:49.515ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35832025-04-16T04:03:49.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35842025-04-16T04:03:49.516ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35852025-04-16T04:03:49.516ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 69.464201ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35862025-04-16T04:03:49.517ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 70.660165ms iteration = 20 status = 0
35872025-04-16T04:03:49.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35882025-04-16T04:03:49.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35892025-04-16T04:03:49.542ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35902025-04-16T04:03:49.542ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35912025-04-16T04:03:49.542ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35922025-04-16T04:03:49.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35932025-04-16T04:03:49.547ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
35942025-04-16T04:03:49.547ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35952025-04-16T04:03:49.547ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b45b08-21fb-4d3e-9154-a94dfae6c4ff", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
35962025-04-16T04:03:49.547ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35972025-04-16T04:03:49.549ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35982025-04-16T04:03:49.550ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35992025-04-16T04:03:49.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
36002025-04-16T04:03:49.552ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36012025-04-16T04:03:49.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36022025-04-16T04:03:49.556ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
36032025-04-16T04:03:49.556ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36042025-04-16T04:03:49.556ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d0aae89-04f1-4208-854c-7a7a4b5c82b7", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36052025-04-16T04:03:49.556ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36062025-04-16T04:03:49.559ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36072025-04-16T04:03:49.565ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
36082025-04-16T04:03:49.565ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36092025-04-16T04:03:49.566ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a54e0294-d373-4517-87f5-d294e69eea2d", "content-length": "26", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36102025-04-16T04:03:49.566ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36112025-04-16T04:03:49.567ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
36122025-04-16T04:03:49.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 121.809124ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
36132025-04-16T04:03:49.590ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36142025-04-16T04:03:49.590ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36152025-04-16T04:03:49.590ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 142.91281ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36162025-04-16T04:03:49.616ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
36172025-04-16T04:03:49.616ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 169.330512ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
36182025-04-16T04:03:49.625ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36192025-04-16T04:03:49.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36202025-04-16T04:03:49.627ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36212025-04-16T04:03:49.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36222025-04-16T04:03:49.629ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36232025-04-16T04:03:49.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36242025-04-16T04:03:49.631ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36252025-04-16T04:03:49.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36262025-04-16T04:03:49.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36272025-04-16T04:03:49.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36282025-04-16T04:03:49.638ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 191.47122ms iteration = 22 status = {"physical_disks_added":0}
36292025-04-16T04:03:49.692ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36302025-04-16T04:03:49.692ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36312025-04-16T04:03:49.693ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbee721e-8412-4e91-a69a-ae1631b77563", "content-length": "372", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36322025-04-16T04:03:49.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36332025-04-16T04:03:49.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36342025-04-16T04:03:49.724ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36352025-04-16T04:03:49.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36362025-04-16T04:03:49.726ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 279.329674ms iteration = 20 status = null
36372025-04-16T04:03:49.736ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
36382025-04-16T04:03:49.736ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
36392025-04-16T04:03:49.736ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "83834f6d-7a3c-44d7-9c03-7dcd0776d13e", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36402025-04-16T04:03:49.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36412025-04-16T04:03:49.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36422025-04-16T04:03:49.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36432025-04-16T04:03:49.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36442025-04-16T04:03:49.743ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 295.905621ms iteration = 20 status = {}
36452025-04-16T04:03:49.777ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
36462025-04-16T04:03:49.777ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "95eee2ea-29c7-45e3-92cc-d1f01c4d00a4", "content-length": "691", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36472025-04-16T04:03:49.777ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
36482025-04-16T04:03:49.777ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
36492025-04-16T04:03:49.777ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
36502025-04-16T04:03:49.779ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 332.82626ms iteration = 20 status = {}
36512025-04-16T04:03:49.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36522025-04-16T04:03:49.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36532025-04-16T04:03:49.783ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
36542025-04-16T04:03:49.783ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36552025-04-16T04:03:49.783ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0c6c2dbf-bf09-44a6-a88f-eb6db902d6e4", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36562025-04-16T04:03:49.785ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 338.870819ms iteration = 20 status = {}
36572025-04-16T04:03:49.803ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36582025-04-16T04:03:49.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36592025-04-16T04:03:49.805ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
36602025-04-16T04:03:49.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36612025-04-16T04:03:49.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36622025-04-16T04:03:49.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dcddd62a-178e-4cdc-9fa8-445764b5718a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36632025-04-16T04:03:49.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dcddd62a-178e-4cdc-9fa8-445764b5718a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36642025-04-16T04:03:49.818ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 7644 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dcddd62a-178e-4cdc-9fa8-445764b5718a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36652025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
36662025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
36672025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
36682025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36692025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36702025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36712025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36722025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36732025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36742025-04-16T04:03:49.821ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36752025-04-16T04:03:49.822ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
36762025-04-16T04:03:49.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36772025-04-16T04:03:49.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36782025-04-16T04:03:49.831ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
36792025-04-16T04:03:49.831ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36802025-04-16T04:03:49.952ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ad93e145-a8a4-412a-8fec-a800cd28b436", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36812025-04-16T04:03:49.952ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
36822025-04-16T04:03:49.952ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36832025-04-16T04:03:49.952ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "46d51a63-dded-493f-b429-c66d70968c2d", "content-length": "682", "date": "Wed, 16 Apr 2025 04:03:49 GMT"} })
36842025-04-16T04:03:49.952ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36852025-04-16T04:03:49.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36862025-04-16T04:03:49.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36872025-04-16T04:03:49.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36882025-04-16T04:03:49.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36892025-04-16T04:03:49.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36902025-04-16T04:03:49.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36912025-04-16T04:03:49.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36922025-04-16T04:03:49.971ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 524.39677ms iteration = 20 status = {}
36932025-04-16T04:03:50.954ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4c5228de-0853-4485-b918-1edb5edd44f1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36942025-04-16T04:03:50.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4c5228de-0853-4485-b918-1edb5edd44f1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36952025-04-16T04:03:50.956ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2601 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4c5228de-0853-4485-b918-1edb5edd44f1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36962025-04-16T04:03:51.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d307827-d6ba-49dd-83a7-9cf7a83b5104 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36972025-04-16T04:03:51.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d307827-d6ba-49dd-83a7-9cf7a83b5104 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36982025-04-16T04:03:51.982ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8d307827-d6ba-49dd-83a7-9cf7a83b5104 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
36992025-04-16T04:03:53.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44469ddd-45c9-4e42-b9cc-216b960554ab roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37002025-04-16T04:03:53.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44469ddd-45c9-4e42-b9cc-216b960554ab resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37012025-04-16T04:03:53.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2250 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44469ddd-45c9-4e42-b9cc-216b960554ab response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37022025-04-16T04:03:54.025ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 72bf7442-739a-462b-b524-488c7f54d418 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37032025-04-16T04:03:54.025ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 72bf7442-739a-462b-b524-488c7f54d418 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37042025-04-16T04:03:54.027ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2132 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 72bf7442-739a-462b-b524-488c7f54d418 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37052025-04-16T04:03:55.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0456c928-71ae-46b4-9f52-838c5829ed88 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37062025-04-16T04:03:55.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0456c928-71ae-46b4-9f52-838c5829ed88 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37072025-04-16T04:03:55.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0456c928-71ae-46b4-9f52-838c5829ed88 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37082025-04-16T04:03:56.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 95ac3055-cece-4f26-a28c-bf8e166afb42 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37092025-04-16T04:03:56.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 95ac3055-cece-4f26-a28c-bf8e166afb42 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37102025-04-16T04:03:56.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2592 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 95ac3055-cece-4f26-a28c-bf8e166afb42 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37112025-04-16T04:03:57.091ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5dbd741-cd94-48b9-935d-124397966f90 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37122025-04-16T04:03:57.091ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5dbd741-cd94-48b9-935d-124397966f90 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37132025-04-16T04:03:57.093ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2493 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5dbd741-cd94-48b9-935d-124397966f90 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37142025-04-16T04:03:58.114ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7ded4c27-538a-4b02-9805-a66ec9052d49 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37152025-04-16T04:03:58.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7ded4c27-538a-4b02-9805-a66ec9052d49 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37162025-04-16T04:03:58.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2066 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7ded4c27-538a-4b02-9805-a66ec9052d49 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37172025-04-16T04:03:58.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 572 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = f65fb448-5ab0-4bf9-96bb-f7297eecb305 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
37182025-04-16T04:03:59.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 92917f0d-743c-4013-99f8-3cf80bdb9174 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37192025-04-16T04:03:59.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 92917f0d-743c-4013-99f8-3cf80bdb9174 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37202025-04-16T04:03:59.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2461 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 92917f0d-743c-4013-99f8-3cf80bdb9174 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37212025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
37222025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
37232025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37242025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37252025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37262025-04-16T04:03:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37272025-04-16T04:03:59.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37282025-04-16T04:03:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37292025-04-16T04:03:59.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37302025-04-16T04:03:59.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37312025-04-16T04:03:59.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37322025-04-16T04:03:59.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37332025-04-16T04:03:59.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37342025-04-16T04:03:59.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
37352025-04-16T04:03:59.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37362025-04-16T04:03:59.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37372025-04-16T04:03:59.464ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
37382025-04-16T04:03:59.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.669619ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
37392025-04-16T04:03:59.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37402025-04-16T04:03:59.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37412025-04-16T04:03:59.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.937713ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37422025-04-16T04:04:00.157ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2675a470-e21b-4c8c-ab9e-4c22f02f1bec roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37432025-04-16T04:04:00.158ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2675a470-e21b-4c8c-ab9e-4c22f02f1bec resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37442025-04-16T04:04:00.160ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2688 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2675a470-e21b-4c8c-ab9e-4c22f02f1bec response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37452025-04-16T04:04:01.179ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2f51a7d6-8682-4384-8f0c-7d572835147a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37462025-04-16T04:04:01.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2f51a7d6-8682-4384-8f0c-7d572835147a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37472025-04-16T04:04:01.181ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1897 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2f51a7d6-8682-4384-8f0c-7d572835147a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37482025-04-16T04:04:02.202ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3af73559-f07a-4468-af4f-1459143b0613 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37492025-04-16T04:04:02.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3af73559-f07a-4468-af4f-1459143b0613 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37502025-04-16T04:04:02.204ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1790 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3af73559-f07a-4468-af4f-1459143b0613 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37512025-04-16T04:04:03.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3885c26b-0ed3-4e0d-9fb3-b8c740cc5f77 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37522025-04-16T04:04:03.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3885c26b-0ed3-4e0d-9fb3-b8c740cc5f77 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37532025-04-16T04:04:03.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2318 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3885c26b-0ed3-4e0d-9fb3-b8c740cc5f77 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37542025-04-16T04:04:04.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5ff6cfd-c236-4e11-af3d-a3173e6827a9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37552025-04-16T04:04:04.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5ff6cfd-c236-4e11-af3d-a3173e6827a9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37562025-04-16T04:04:04.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2570 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5ff6cfd-c236-4e11-af3d-a3173e6827a9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37572025-04-16T04:04:05.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cbbed261-920e-46e8-bd7f-9f6614ae4f99 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37582025-04-16T04:04:05.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cbbed261-920e-46e8-bd7f-9f6614ae4f99 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37592025-04-16T04:04:05.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2493 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cbbed261-920e-46e8-bd7f-9f6614ae4f99 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37602025-04-16T04:04:06.292ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e0009298-31dd-43ca-844a-990f449c73c6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37612025-04-16T04:04:06.292ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e0009298-31dd-43ca-844a-990f449c73c6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37622025-04-16T04:04:06.294ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2146 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e0009298-31dd-43ca-844a-990f449c73c6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37632025-04-16T04:04:07.316ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e8480fc-581e-497a-b375-c9f0aa4b06be roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37642025-04-16T04:04:07.316ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e8480fc-581e-497a-b375-c9f0aa4b06be resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37652025-04-16T04:04:07.318ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1835 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9e8480fc-581e-497a-b375-c9f0aa4b06be response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37662025-04-16T04:04:08.338ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bfaf5e0f-37f1-4b10-af3d-91c065376b7c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37672025-04-16T04:04:08.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bfaf5e0f-37f1-4b10-af3d-91c065376b7c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37682025-04-16T04:04:08.340ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bfaf5e0f-37f1-4b10-af3d-91c065376b7c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37692025-04-16T04:04:08.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 648 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 3ff17525-eeac-48c1-9bcc-1e6a5658d7d2 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
37702025-04-16T04:04:09.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a57f16b-d6b5-4c72-a048-ad98f2fd14cd roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37712025-04-16T04:04:09.361ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a57f16b-d6b5-4c72-a048-ad98f2fd14cd resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37722025-04-16T04:04:09.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2339 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a57f16b-d6b5-4c72-a048-ad98f2fd14cd response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37732025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
37742025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
37752025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37762025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37772025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37782025-04-16T04:04:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37792025-04-16T04:04:09.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37802025-04-16T04:04:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37812025-04-16T04:04:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37822025-04-16T04:04:09.451ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.376316ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37832025-04-16T04:04:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37842025-04-16T04:04:09.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37852025-04-16T04:04:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37862025-04-16T04:04:09.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37872025-04-16T04:04:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37882025-04-16T04:04:09.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37892025-04-16T04:04:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
37902025-04-16T04:04:09.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37912025-04-16T04:04:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37922025-04-16T04:04:09.471ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
37932025-04-16T04:04:09.471ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.027434ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
37942025-04-16T04:04:10.383ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b50a81-5648-41ac-baca-8f53478d711c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37952025-04-16T04:04:10.383ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b50a81-5648-41ac-baca-8f53478d711c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37962025-04-16T04:04:10.385ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2092 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b50a81-5648-41ac-baca-8f53478d711c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37972025-04-16T04:04:11.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 114d8077-f67f-40f2-a042-ae55c17d4601 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37982025-04-16T04:04:11.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 114d8077-f67f-40f2-a042-ae55c17d4601 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
37992025-04-16T04:04:11.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1957 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 114d8077-f67f-40f2-a042-ae55c17d4601 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38002025-04-16T04:04:12.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 48216514-4a15-4737-a8f0-ab568e9d1ac3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38012025-04-16T04:04:12.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 48216514-4a15-4737-a8f0-ab568e9d1ac3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38022025-04-16T04:04:12.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 48216514-4a15-4737-a8f0-ab568e9d1ac3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38032025-04-16T04:04:13.451ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7e11119b-29fa-43aa-869a-624ca74d645e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38042025-04-16T04:04:13.451ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7e11119b-29fa-43aa-869a-624ca74d645e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38052025-04-16T04:04:13.453ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7e11119b-29fa-43aa-869a-624ca74d645e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38062025-04-16T04:04:14.474ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88ea4c75-40d1-47e9-a592-99bf56028c6e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38072025-04-16T04:04:14.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88ea4c75-40d1-47e9-a592-99bf56028c6e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38082025-04-16T04:04:14.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 88ea4c75-40d1-47e9-a592-99bf56028c6e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38092025-04-16T04:04:15.496ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 13b378ae-3c87-4711-a067-35e7c4968c64 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38102025-04-16T04:04:15.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 13b378ae-3c87-4711-a067-35e7c4968c64 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38112025-04-16T04:04:15.499ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 13b378ae-3c87-4711-a067-35e7c4968c64 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38122025-04-16T04:04:16.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d19e4d13-ed9c-41a4-a891-6dd45bbc4d9e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38132025-04-16T04:04:16.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d19e4d13-ed9c-41a4-a891-6dd45bbc4d9e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38142025-04-16T04:04:16.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4358 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d19e4d13-ed9c-41a4-a891-6dd45bbc4d9e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38152025-04-16T04:04:17.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 800db67d-7632-4009-a972-b640b5e1388f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38162025-04-16T04:04:17.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 800db67d-7632-4009-a972-b640b5e1388f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38172025-04-16T04:04:17.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2511 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 800db67d-7632-4009-a972-b640b5e1388f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38182025-04-16T04:04:18.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 489 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = db593435-d22c-4aec-8700-797cc2c63fb0 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
38192025-04-16T04:04:18.567ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28b88513-8586-4e79-8e9a-937359712780 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38202025-04-16T04:04:18.567ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28b88513-8586-4e79-8e9a-937359712780 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38212025-04-16T04:04:18.569ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2133 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28b88513-8586-4e79-8e9a-937359712780 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
38222025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
38232025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
38242025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38252025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
38262025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
38272025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
38282025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
38292025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38302025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
38312025-04-16T04:04:19.446ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
38322025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
38332025-04-16T04:04:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
38342025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.34\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
38352025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
38362025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
38372025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38382025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
38392025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38402025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
38412025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38422025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38432025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
38442025-04-16T04:04:19.446ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
38452025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38462025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
38472025-04-16T04:04:19.446ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
38482025-04-16T04:04:19.446ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
38492025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
38502025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
38512025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
38522025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
38532025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
38542025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38552025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
38562025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
38572025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
38582025-04-16T04:04:19.446ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
38592025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38602025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
38612025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
38622025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38632025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
38642025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
38652025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
38662025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38672025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Timeout
38682025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
38692025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
38702025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
38712025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38722025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
38732025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
38742025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
38752025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38762025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
38772025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
38782025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
38792025-04-16T04:04:19.446ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:54:19.446825554 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
38802025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38812025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
38822025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
38832025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
38842025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38852025-04-16T04:04:19.446ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38862025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
38872025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
38882025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
38892025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38902025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
38912025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38922025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
38932025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
38942025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
38952025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
38962025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38972025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38982025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
38992025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
39002025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39012025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39022025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39032025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
39042025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39052025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39062025-04-16T04:04:19.447ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
39072025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39082025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
39092025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
39102025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39112025-04-16T04:04:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3149, tv_nsec: 727839064 } })
39122025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39132025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39142025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39152025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39162025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
39172025-04-16T04:04:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3149, tv_nsec: 727913665 } })
39182025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39192025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
39202025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39212025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39222025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39232025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39242025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39252025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39262025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
39272025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
39282025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39292025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39302025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
39312025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 999.589\u{b5}s iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
39322025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
39332025-04-16T04:04:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.433129ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
39342025-04-16T04:04:19.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39352025-04-16T04:04:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39362025-04-16T04:04:19.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.112631ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
39372025-04-16T04:04:19.449ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.911327ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
39382025-04-16T04:04:19.450ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39392025-04-16T04:04:19.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39402025-04-16T04:04:19.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39412025-04-16T04:04:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39422025-04-16T04:04:19.452ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39432025-04-16T04:04:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39442025-04-16T04:04:19.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39452025-04-16T04:04:19.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39462025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39472025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39482025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39492025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39502025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39512025-04-16T04:04:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39522025-04-16T04:04:19.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39532025-04-16T04:04:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39542025-04-16T04:04:19.459ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39552025-04-16T04:04:19.463ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
39562025-04-16T04:04:19.463ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
39572025-04-16T04:04:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
39582025-04-16T04:04:19.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39592025-04-16T04:04:19.464ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39602025-04-16T04:04:19.464ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
39612025-04-16T04:04:19.464ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
39622025-04-16T04:04:19.464ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 17.614538ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
39632025-04-16T04:04:19.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
39642025-04-16T04:04:19.466ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
39652025-04-16T04:04:19.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39662025-04-16T04:04:19.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
39672025-04-16T04:04:19.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
39682025-04-16T04:04:19.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39692025-04-16T04:04:19.468ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39702025-04-16T04:04:19.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39712025-04-16T04:04:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39722025-04-16T04:04:19.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39732025-04-16T04:04:19.469ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39742025-04-16T04:04:19.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39752025-04-16T04:04:19.470ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:54:19.446825554 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
39762025-04-16T04:04:19.470ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 23.268989ms iteration = 11 status = {"expiration":"2025-04-16T03:54:19.446825554Z","pruned":[]}
39772025-04-16T04:04:19.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39782025-04-16T04:04:19.471ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39792025-04-16T04:04:19.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39802025-04-16T04:04:19.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
39812025-04-16T04:04:19.472ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39822025-04-16T04:04:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39832025-04-16T04:04:19.473ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39842025-04-16T04:04:19.474ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3149, tv_nsec: 754888028 } })
39852025-04-16T04:04:19.474ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
39862025-04-16T04:04:19.474ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 27.88457ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
39872025-04-16T04:04:19.475ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39882025-04-16T04:04:19.475ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3149, tv_nsec: 756199454 } })
39892025-04-16T04:04:19.475ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
39902025-04-16T04:04:19.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39912025-04-16T04:04:19.477ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39922025-04-16T04:04:19.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39932025-04-16T04:04:19.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
39942025-04-16T04:04:19.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39952025-04-16T04:04:19.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39962025-04-16T04:04:19.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39972025-04-16T04:04:19.483ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3149, tv_nsec: 764085329 } })
39982025-04-16T04:04:19.483ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
39992025-04-16T04:04:19.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40002025-04-16T04:04:19.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40012025-04-16T04:04:19.484ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40022025-04-16T04:04:19.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40032025-04-16T04:04:19.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40042025-04-16T04:04:19.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40052025-04-16T04:04:19.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40062025-04-16T04:04:19.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40072025-04-16T04:04:19.486ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40082025-04-16T04:04:19.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
40092025-04-16T04:04:19.489ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
40102025-04-16T04:04:19.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
40112025-04-16T04:04:19.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40122025-04-16T04:04:19.491ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40132025-04-16T04:04:19.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40142025-04-16T04:04:19.491ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40152025-04-16T04:04:19.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40162025-04-16T04:04:19.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40172025-04-16T04:04:19.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40182025-04-16T04:04:19.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40192025-04-16T04:04:19.494ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40202025-04-16T04:04:19.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40212025-04-16T04:04:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40222025-04-16T04:04:19.495ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40232025-04-16T04:04:19.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40242025-04-16T04:04:19.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40252025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
40262025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
40272025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
40282025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40292025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40302025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
40312025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40322025-04-16T04:04:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40332025-04-16T04:04:19.499ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40342025-04-16T04:04:19.499ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40352025-04-16T04:04:19.499ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40362025-04-16T04:04:19.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40372025-04-16T04:04:19.501ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40382025-04-16T04:04:19.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40392025-04-16T04:04:19.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40402025-04-16T04:04:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40412025-04-16T04:04:19.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.371881ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
40422025-04-16T04:04:19.511ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40432025-04-16T04:04:19.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40442025-04-16T04:04:19.512ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 65.527403ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
40452025-04-16T04:04:19.519ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40462025-04-16T04:04:19.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
40472025-04-16T04:04:19.522ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40482025-04-16T04:04:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40492025-04-16T04:04:19.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40502025-04-16T04:04:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40512025-04-16T04:04:19.523ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
40522025-04-16T04:04:19.523ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
40532025-04-16T04:04:19.523ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.318077ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
40542025-04-16T04:04:19.524ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.376256ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40552025-04-16T04:04:19.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
40562025-04-16T04:04:19.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
40572025-04-16T04:04:19.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
40582025-04-16T04:04:19.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40592025-04-16T04:04:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40602025-04-16T04:04:19.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40612025-04-16T04:04:19.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40622025-04-16T04:04:19.535ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40632025-04-16T04:04:19.535ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 88.486177ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40642025-04-16T04:04:19.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
40652025-04-16T04:04:19.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
40662025-04-16T04:04:19.564ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
40672025-04-16T04:04:19.591ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d029da25-1d8e-4fd4-a695-f8faaec4de19 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
40682025-04-16T04:04:19.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d029da25-1d8e-4fd4-a695-f8faaec4de19 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
40692025-04-16T04:04:19.598ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
40702025-04-16T04:04:19.598ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 152.129123ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
40712025-04-16T04:04:19.608ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
40722025-04-16T04:04:19.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40732025-04-16T04:04:19.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40742025-04-16T04:04:19.642ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
40752025-04-16T04:04:19.643ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40762025-04-16T04:04:19.643ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bc92df0-994b-4ffc-bb63-8860477ea042", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
40772025-04-16T04:04:19.643ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40782025-04-16T04:04:19.644ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40792025-04-16T04:04:19.678ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
40802025-04-16T04:04:19.678ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40812025-04-16T04:04:19.678ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0999a95-818f-44b6-9164-b7d3bdd9d8c7", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
40822025-04-16T04:04:19.678ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40832025-04-16T04:04:19.680ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40842025-04-16T04:04:19.857ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
40852025-04-16T04:04:19.857ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40862025-04-16T04:04:19.858ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e0a20d8-f6e1-42b6-90e9-71028f26d0d4", "content-length": "372", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
40872025-04-16T04:04:19.858ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40882025-04-16T04:04:19.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40892025-04-16T04:04:19.861ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40902025-04-16T04:04:19.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40912025-04-16T04:04:19.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40922025-04-16T04:04:19.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40932025-04-16T04:04:19.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40942025-04-16T04:04:19.866ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40952025-04-16T04:04:19.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40962025-04-16T04:04:19.867ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 420.259414ms iteration = 21 status = {}
40972025-04-16T04:04:19.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40982025-04-16T04:04:19.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40992025-04-16T04:04:19.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41002025-04-16T04:04:19.875ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41012025-04-16T04:04:19.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41022025-04-16T04:04:19.876ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
41032025-04-16T04:04:19.876ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41042025-04-16T04:04:19.877ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db94e80-9b09-46b6-930b-9b18101edc49", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
41052025-04-16T04:04:19.877ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41062025-04-16T04:04:19.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41072025-04-16T04:04:19.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41082025-04-16T04:04:19.879ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41092025-04-16T04:04:19.879ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
41102025-04-16T04:04:19.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41112025-04-16T04:04:19.880ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
41122025-04-16T04:04:19.880ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 433.876693ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
41132025-04-16T04:04:19.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 288542 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d029da25-1d8e-4fd4-a695-f8faaec4de19 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
41142025-04-16T04:04:19.880ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41152025-04-16T04:04:19.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41162025-04-16T04:04:19.884ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41172025-04-16T04:04:19.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41182025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41192025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41202025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41212025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41222025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41232025-04-16T04:04:19.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41242025-04-16T04:04:19.888ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
41252025-04-16T04:04:19.888ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "35197056-03af-4efb-918b-84ef2b6f04ad", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41262025-04-16T04:04:19.888ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41272025-04-16T04:04:19.890ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41282025-04-16T04:04:19.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41292025-04-16T04:04:19.893ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "64e1e0cd-f3f6-4e34-b101-d079d7c87855", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41302025-04-16T04:04:19.893ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41312025-04-16T04:04:19.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41322025-04-16T04:04:19.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41332025-04-16T04:04:19.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
41342025-04-16T04:04:19.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41352025-04-16T04:04:19.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41362025-04-16T04:04:19.896ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41372025-04-16T04:04:19.896ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 450.054962ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
41382025-04-16T04:04:19.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41392025-04-16T04:04:19.903ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41402025-04-16T04:04:19.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41412025-04-16T04:04:19.906ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41422025-04-16T04:04:19.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41432025-04-16T04:04:19.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41442025-04-16T04:04:19.918ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 471.067767ms iteration = 21 status = 0
41452025-04-16T04:04:19.925ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
41462025-04-16T04:04:19.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41472025-04-16T04:04:19.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41482025-04-16T04:04:19.926ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41492025-04-16T04:04:19.926ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
41502025-04-16T04:04:19.926ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41512025-04-16T04:04:19.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41522025-04-16T04:04:19.927ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 481.089445ms iteration = 21 status = null
41532025-04-16T04:04:19.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41542025-04-16T04:04:19.931ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
41552025-04-16T04:04:19.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41562025-04-16T04:04:19.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41572025-04-16T04:04:19.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41582025-04-16T04:04:19.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41592025-04-16T04:04:19.937ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 490.47382ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41602025-04-16T04:04:19.942ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 495.793126ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41612025-04-16T04:04:19.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41622025-04-16T04:04:19.942ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
41632025-04-16T04:04:19.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
41642025-04-16T04:04:19.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
41652025-04-16T04:04:19.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41662025-04-16T04:04:19.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41672025-04-16T04:04:19.947ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
41682025-04-16T04:04:19.950ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
41692025-04-16T04:04:19.950ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41702025-04-16T04:04:19.950ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41712025-04-16T04:04:19.950ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41722025-04-16T04:04:19.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41732025-04-16T04:04:19.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41742025-04-16T04:04:19.956ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
41752025-04-16T04:04:19.956ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
41762025-04-16T04:04:19.956ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41772025-04-16T04:04:19.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41782025-04-16T04:04:19.957ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 511.184239ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
41792025-04-16T04:04:19.958ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41802025-04-16T04:04:19.958ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41812025-04-16T04:04:19.959ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41822025-04-16T04:04:19.961ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 515.132608ms iteration = 23 status = {"physical_disks_added":0}
41832025-04-16T04:04:19.962ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41842025-04-16T04:04:19.962ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41852025-04-16T04:04:19.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41862025-04-16T04:04:19.964ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41872025-04-16T04:04:19.964ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41882025-04-16T04:04:19.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41892025-04-16T04:04:19.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41902025-04-16T04:04:19.965ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41912025-04-16T04:04:19.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41922025-04-16T04:04:19.971ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
41932025-04-16T04:04:19.971ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
41942025-04-16T04:04:19.971ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
41952025-04-16T04:04:19.971ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 524.970021ms iteration = 11 status = {"generation":1}
41962025-04-16T04:04:19.976ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41972025-04-16T04:04:19.976ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
41982025-04-16T04:04:19.976ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 529.371838ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
41992025-04-16T04:04:19.977ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
42002025-04-16T04:04:19.977ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
42012025-04-16T04:04:19.977ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "28801ded-a8cf-4154-8562-ffe7457b5e9f", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
42022025-04-16T04:04:19.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42032025-04-16T04:04:19.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42042025-04-16T04:04:19.983ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
42052025-04-16T04:04:19.983ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 536.948758ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
42062025-04-16T04:04:19.985ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42072025-04-16T04:04:19.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42082025-04-16T04:04:19.995ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42092025-04-16T04:04:19.996ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-04-16T04:04:19.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
42112025-04-16T04:04:19.998ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
42122025-04-16T04:04:19.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
42132025-04-16T04:04:19.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42142025-04-16T04:04:20.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
42152025-04-16T04:04:20.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42162025-04-16T04:04:20.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42172025-04-16T04:04:20.000ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
42182025-04-16T04:04:20.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
42192025-04-16T04:04:20.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
42202025-04-16T04:04:20.011ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
42212025-04-16T04:04:20.011ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 564.577443ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
42222025-04-16T04:04:20.092ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
42232025-04-16T04:04:20.092ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a10db24e-c63f-4b1a-9e90-9a238b341923", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42242025-04-16T04:04:20.092ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42252025-04-16T04:04:20.094ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b9f3cc99-00ad-4b9d-addd-db8a71c82ee9", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42262025-04-16T04:04:20.094ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42272025-04-16T04:04:20.096ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "91ab6349-c1af-4247-915d-3ae5c68488fc", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42282025-04-16T04:04:20.096ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42292025-04-16T04:04:20.097ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
42302025-04-16T04:04:20.097ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 651.374837ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
42312025-04-16T04:04:20.122ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42322025-04-16T04:04:20.122ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fcb9449-695d-40c4-bd54-808a050a1f1d", "content-length": "691", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
42332025-04-16T04:04:20.122ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
42342025-04-16T04:04:20.122ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
42352025-04-16T04:04:20.122ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
42362025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 677.677625ms iteration = 21 status = {}
42372025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
42382025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
42392025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
42402025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 677.844169ms iteration = 11 status = {"generation":2}
42412025-04-16T04:04:20.124ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42422025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42432025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
42442025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42452025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
42462025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42472025-04-16T04:04:20.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42482025-04-16T04:04:20.126ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42492025-04-16T04:04:20.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42502025-04-16T04:04:20.127ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
42512025-04-16T04:04:20.127ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42522025-04-16T04:04:20.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42532025-04-16T04:04:20.127ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
42542025-04-16T04:04:20.127ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 681.201706ms iteration = 5 status = {"blueprint_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","inventory_collection_id":"42df9c3a-585e-4d43-8ee6-6c8ce980b0b3","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
42552025-04-16T04:04:20.128ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42562025-04-16T04:04:20.129ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42572025-04-16T04:04:20.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42582025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
42592025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
42602025-04-16T04:04:20.130ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
42612025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42622025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
42632025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42642025-04-16T04:04:20.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42652025-04-16T04:04:20.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42662025-04-16T04:04:20.132ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42672025-04-16T04:04:20.132ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
42682025-04-16T04:04:20.133ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
42692025-04-16T04:04:20.133ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 686.637373ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
42702025-04-16T04:04:20.134ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "af23a799-bdfe-4775-8cb1-4de45b5b662e", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
42712025-04-16T04:04:20.134ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
42722025-04-16T04:04:20.134ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "73144959-22c9-4911-8968-38b53c05f7a3", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
42732025-04-16T04:04:20.134ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
42742025-04-16T04:04:20.135ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
42752025-04-16T04:04:20.135ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42762025-04-16T04:04:20.136ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5960f617-f14d-49f4-a5a3-ab51f405b4b9", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
42772025-04-16T04:04:20.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
42782025-04-16T04:04:20.136ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42792025-04-16T04:04:20.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42802025-04-16T04:04:20.136ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 690.151512ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T03:54:20.204201558Z"},{"operation":"list","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T03:54:20.240568726Z"},{"generation":1,"operation":"put_config","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T03:59:19.876409863Z"},{"operation":"list","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T03:59:19.876735939Z"},{"generation":1,"operation":"put_config","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T04:04:20.134555568Z"},{"operation":"list","target_sled":"fe2a235d-b62d-4168-a36a-687174761bf3","time":"2025-04-16T04:04:20.134950776Z"}]}
42812025-04-16T04:04:20.138ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 691.743563ms iteration = 21 status = {}
42822025-04-16T04:04:20.139ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42832025-04-16T04:04:20.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42842025-04-16T04:04:20.141ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42852025-04-16T04:04:20.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42862025-04-16T04:04:20.142ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42872025-04-16T04:04:20.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42882025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
42892025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
42902025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
42912025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
42922025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42932025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42942025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42952025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42962025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42972025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42982025-04-16T04:04:20.143ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
42992025-04-16T04:04:20.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43002025-04-16T04:04:20.144ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43012025-04-16T04:04:20.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43022025-04-16T04:04:20.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43032025-04-16T04:04:20.145ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43042025-04-16T04:04:20.146ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
43052025-04-16T04:04:20.146ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
43062025-04-16T04:04:20.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43072025-04-16T04:04:20.147ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43082025-04-16T04:04:20.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43092025-04-16T04:04:20.148ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43102025-04-16T04:04:20.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
43112025-04-16T04:04:20.152ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43122025-04-16T04:04:20.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43132025-04-16T04:04:20.153ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43142025-04-16T04:04:20.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43152025-04-16T04:04:20.156ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43162025-04-16T04:04:20.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43172025-04-16T04:04:20.159ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43182025-04-16T04:04:20.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43192025-04-16T04:04:20.161ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43202025-04-16T04:04:20.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43212025-04-16T04:04:20.162ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43222025-04-16T04:04:20.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43232025-04-16T04:04:20.164ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43242025-04-16T04:04:20.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43252025-04-16T04:04:20.166ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43262025-04-16T04:04:20.170ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
43272025-04-16T04:04:20.170ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43282025-04-16T04:04:20.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43292025-04-16T04:04:20.171ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43302025-04-16T04:04:20.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43312025-04-16T04:04:20.173ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43322025-04-16T04:04:20.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43332025-04-16T04:04:20.174ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43342025-04-16T04:04:20.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
43352025-04-16T04:04:20.178ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43362025-04-16T04:04:20.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43372025-04-16T04:04:20.185ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43382025-04-16T04:04:20.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43392025-04-16T04:04:20.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43402025-04-16T04:04:20.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43412025-04-16T04:04:20.189ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43422025-04-16T04:04:20.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43432025-04-16T04:04:20.191ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43442025-04-16T04:04:20.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43452025-04-16T04:04:20.194ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43462025-04-16T04:04:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43472025-04-16T04:04:20.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43482025-04-16T04:04:20.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43492025-04-16T04:04:20.196ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43502025-04-16T04:04:20.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43512025-04-16T04:04:20.198ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43522025-04-16T04:04:20.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
43532025-04-16T04:04:20.202ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43542025-04-16T04:04:20.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43552025-04-16T04:04:20.203ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43562025-04-16T04:04:20.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43572025-04-16T04:04:20.205ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43582025-04-16T04:04:20.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43592025-04-16T04:04:20.206ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43602025-04-16T04:04:20.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
43612025-04-16T04:04:20.209ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43622025-04-16T04:04:20.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43632025-04-16T04:04:20.227ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 30d28304-b013-4f5e-b883-f768d780b244, kind: Service { id: 97388ba5-45a6-4ce2-ac1c-18f5fc13fe23 }, name: Name("external-dns-97388ba5-45a6-4ce2-ac1c-18f5fc13fe23"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 59])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ed64c2fe-6b0b-4760-a321-1f51f9268300, kind: Service { id: ead8dc35-753d-459f-bec4-f8ca9d19736f }, name: Name("external-dns-ead8dc35-753d-459f-bec4-f8ca9d19736f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 163, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 249dc19b-4d14-4383-a993-73d4e41fbf66, kind: Service { id: 54fda60b-b51d-498f-9226-2a20c82f92f0 }, name: Name("nexus-54fda60b-b51d-498f-9226-2a20c82f92f0"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7abf494b-a16b-4ddf-bed1-babe54af3b2c, kind: Service { id: cf81244d-0a39-4112-8f8e-0d08c72b054b }, name: Name("nexus-cf81244d-0a39-4112-8f8e-0d08c72b054b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 231])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ffe3b08b-c832-4273-8803-cd2b71fbf397, kind: Service { id: 9fa72b7d-ab32-4040-b078-1b91700ee410 }, name: Name("nexus-9fa72b7d-ab32-4040-b078-1b91700ee410"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
43642025-04-16T04:04:20.227ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43652025-04-16T04:04:20.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43662025-04-16T04:04:20.235ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43672025-04-16T04:04:20.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
43682025-04-16T04:04:20.237ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43692025-04-16T04:04:20.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43702025-04-16T04:04:20.239ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
43712025-04-16T04:04:20.239ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 30d28304-b013-4f5e-b883-f768d780b244, kind: Service { id: 97388ba5-45a6-4ce2-ac1c-18f5fc13fe23 }, name: Name("external-dns-97388ba5-45a6-4ce2-ac1c-18f5fc13fe23"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 59])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ed64c2fe-6b0b-4760-a321-1f51f9268300, kind: Service { id: ead8dc35-753d-459f-bec4-f8ca9d19736f }, name: Name("external-dns-ead8dc35-753d-459f-bec4-f8ca9d19736f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 163, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 249dc19b-4d14-4383-a993-73d4e41fbf66, kind: Service { id: 54fda60b-b51d-498f-9226-2a20c82f92f0 }, name: Name("nexus-54fda60b-b51d-498f-9226-2a20c82f92f0"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 235])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7abf494b-a16b-4ddf-bed1-babe54af3b2c, kind: Service { id: cf81244d-0a39-4112-8f8e-0d08c72b054b }, name: Name("nexus-cf81244d-0a39-4112-8f8e-0d08c72b054b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 231])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ffe3b08b-c832-4273-8803-cd2b71fbf397, kind: Service { id: 9fa72b7d-ab32-4040-b078-1b91700ee410 }, name: Name("nexus-9fa72b7d-ab32-4040-b078-1b91700ee410"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
43722025-04-16T04:04:20.239ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
43732025-04-16T04:04:20.258ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bc0b523d-491a-4a43-ac58-eda6cbf0e831", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
43742025-04-16T04:04:20.258ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
43752025-04-16T04:04:20.259ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43762025-04-16T04:04:20.259ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7f22f43-5d33-4184-85cf-bdfc29bda2ed", "content-length": "682", "date": "Wed, 16 Apr 2025 04:04:19 GMT"} })
43772025-04-16T04:04:20.259ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43782025-04-16T04:04:20.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43792025-04-16T04:04:20.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43802025-04-16T04:04:20.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43812025-04-16T04:04:20.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43822025-04-16T04:04:20.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43832025-04-16T04:04:20.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43842025-04-16T04:04:20.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43852025-04-16T04:04:20.278ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 831.724697ms iteration = 21 status = {}
43862025-04-16T04:04:20.375ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: fe2a235d-b62d-4168-a36a-687174761bf3, time_created: 2025-04-16T03:54:17.397611Z, time_modified: 2025-04-16T03:54:17.397611Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
43872025-04-16T04:04:20.375ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
43882025-04-16T04:04:20.375ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43892025-04-16T04:04:20.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43902025-04-16T04:04:20.378ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43912025-04-16T04:04:20.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43922025-04-16T04:04:20.379ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43932025-04-16T04:04:20.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
43942025-04-16T04:04:20.415ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
43952025-04-16T04:04:20.415ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "0849f613-c9c5-453e-97ea-70b8d54f12e0", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
43962025-04-16T04:04:20.416ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
43972025-04-16T04:04:20.416ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 970.411934ms
43982025-04-16T04:04:20.416ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 970.422914ms iteration = 3 status = {}
43992025-04-16T04:04:20.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f7467f69-ac0b-43b7-96c6-560497682252 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
44002025-04-16T04:04:20.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f7467f69-ac0b-43b7-96c6-560497682252 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
44012025-04-16T04:04:20.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2736 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f7467f69-ac0b-43b7-96c6-560497682252 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
44022025-04-16T04:04:21.240ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
44032025-04-16T04:04:21.240ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44042025-04-16T04:04:21.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44052025-04-16T04:04:21.243ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44062025-04-16T04:04:21.245ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
44072025-04-16T04:04:21.245ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44082025-04-16T04:04:21.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44092025-04-16T04:04:21.248ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(32528a85-0b55-49bd-a790-d2cada4a9d61, 0), (96df7060-474a-4667-a019-ec10f78d4286, 0), (42df9c3a-585e-4d43-8ee6-6c8ce980b0b3, 0), (eb5e264e-4236-4abe-a5e5-e1df25ccc892, 0)] collection_id = 32528a85-0b55-49bd-a790-d2cada4a9d61
44102025-04-16T04:04:21.248ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44112025-04-16T04:04:21.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44122025-04-16T04:04:21.337ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 32528a85-0b55-49bd-a790-d2cada4a9d61 file = nexus/db-queries/src/db/datastore/inventory.rs:1446 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
44132025-04-16T04:04:21.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44142025-04-16T04:04:21.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44152025-04-16T04:04:21.350ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(96df7060-474a-4667-a019-ec10f78d4286, 0), (42df9c3a-585e-4d43-8ee6-6c8ce980b0b3, 0), (eb5e264e-4236-4abe-a5e5-e1df25ccc892, 0), (b093707e-2324-4a3d-9797-adf2c81c8093, 0)] collection_id = 96df7060-474a-4667-a019-ec10f78d4286
44162025-04-16T04:04:21.350ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44172025-04-16T04:04:21.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44182025-04-16T04:04:21.405ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 96df7060-474a-4667-a019-ec10f78d4286 file = nexus/db-queries/src/db/datastore/inventory.rs:1446 ncabooses = 12 nclickhouse_keeper_membership = 0 ncollections = 1 ndatasets = 84 nerrors = 0 nnics = 6 nnvme_disk_firmware = 11 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agent_zones = 1 nsled_agents = 1 nsps = 2 nzones = 28 nzpools = 9
44192025-04-16T04:04:21.405ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44202025-04-16T04:04:21.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44212025-04-16T04:04:21.409ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(42df9c3a-585e-4d43-8ee6-6c8ce980b0b3, 0), (eb5e264e-4236-4abe-a5e5-e1df25ccc892, 0), (b093707e-2324-4a3d-9797-adf2c81c8093, 0)]
44222025-04-16T04:04:21.410ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3151, tv_nsec: 690820729 } })
44232025-04-16T04:04:21.441ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
44242025-04-16T04:04:21.441ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
44252025-04-16T04:04:21.441ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
44262025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2deff88-1205-4ed2-b25b-351c4a970861", "content-length": "476", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44272025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
44282025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b37a2adb-72fa-40aa-a0be-7ac186f50937", "content-length": "734", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44292025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
44302025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25a2a4cd-38cb-423d-a344-f9884b10c94c", "content-length": "111", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44312025-04-16T04:04:21.442ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
44322025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c34ef605-2ad7-4e16-835e-a478192b6be3", "content-length": "111", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44332025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
44342025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c35e87c-0726-49f0-8590-25d1b2855961", "content-length": "167", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44352025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
44362025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d3f7735-c73d-4868-b4bd-39813125f16d", "content-length": "167", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44372025-04-16T04:04:21.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
44382025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "edfd3f50-a82c-49d0-857c-991181e83f0f", "content-length": "175", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44392025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
44402025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b533c60f-2a09-4dea-8299-9ec13082c7e6", "content-length": "175", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44412025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
44422025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fa2dbc3-6ff9-4d8b-8842-1016f666d874", "content-length": "702", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44432025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
44442025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a90b8a75-b736-491a-b9a7-4370609ba464", "content-length": "727", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44452025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
44462025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ded3c5-f249-4c03-bcc5-ff0f67a39f98", "content-length": "729", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44472025-04-16T04:04:21.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
44482025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71d677e9-2cc4-45fb-b79b-115c1f55a8ba", "content-length": "728", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44492025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
44502025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55686754-1530-43b6-ae5b-0cfcc5712c60", "content-length": "722", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44512025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
44522025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab08a1f4-10f0-472f-acda-bd17d8d80f8d", "content-length": "109", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44532025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
44542025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f429f8cc-62fe-4661-95f8-33a06dffb7ce", "content-length": "109", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44552025-04-16T04:04:21.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
44562025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7e5f92d-619f-4654-b44b-a9ccb8932682", "content-length": "169", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44572025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
44582025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "759a97e3-594e-4635-9fab-455a6ae632bc", "content-length": "169", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44592025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
44602025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b2b5d3-e94a-4049-82e1-1cecb7a4ebed", "content-length": "178", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44612025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
44622025-04-16T04:04:21.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e188f8d-eaff-4476-9551-8281a0c4a22f", "content-length": "178", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44632025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
44642025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c822564-4f62-4375-a1d2-189aa0bbd140", "content-length": "702", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44652025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
44662025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27bb7d25-72d7-45ed-a5e5-d942958e273a", "content-length": "727", "date": "Wed, 16 Apr 2025 04:04:20 GMT"} })
44672025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
44682025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "67c39bf1-ab61-496d-af11-50b339443473", "content-length": "729", "date": "Wed, 16 Apr 2025 04:04:21 GMT"} })
44692025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
44702025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f60e3578-d2ae-40f2-a1cf-62c830d81b42", "content-length": "728", "date": "Wed, 16 Apr 2025 04:04:21 GMT"} })
44712025-04-16T04:04:21.449ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44722025-04-16T04:04:21.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44732025-04-16T04:04:21.451ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44742025-04-16T04:04:21.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
44752025-04-16T04:04:21.453ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44762025-04-16T04:04:21.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44772025-04-16T04:04:21.455ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44782025-04-16T04:04:21.457ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
44792025-04-16T04:04:21.457ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44802025-04-16T04:04:21.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44812025-04-16T04:04:21.490ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
44822025-04-16T04:04:21.490ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
44832025-04-16T04:04:21.700ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdf2bc0f-c8ba-4c48-a812-3cf52db320fa", "content-length": "31971", "date": "Wed, 16 Apr 2025 04:04:21 GMT"} })
44842025-04-16T04:04:21.702ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
44852025-04-16T04:04:21.702ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
44862025-04-16T04:04:21.702ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
44872025-04-16T04:04:21.704ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44882025-04-16T04:04:21.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44892025-04-16T04:04:21.706ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44902025-04-16T04:04:21.708ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
44912025-04-16T04:04:21.708ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44922025-04-16T04:04:21.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
44932025-04-16T04:04:21.837ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 9371a2fa-681c-4714-9050-e34ced50da5a file = nexus/db-queries/src/db/datastore/inventory.rs:1009
44942025-04-16T04:04:21.837ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 9371a2fa-681c-4714-9050-e34ced50da5a time_started = 2025-04-16 04:04:21.441643 UTC
44952025-04-16T04:04:21.837ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
44962025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44972025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 597.27341ms iteration = 3 status = {"collection_id":"9371a2fa-681c-4714-9050-e34ced50da5a","time_done":"2025-04-16 04:04:21.702455 UTC","time_started":"2025-04-16 04:04:21.441643 UTC"}
44982025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
44992025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45002025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45012025-04-16T04:04:21.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45022025-04-16T04:04:21.840ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45032025-04-16T04:04:21.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45042025-04-16T04:04:21.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45052025-04-16T04:04:21.842ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45062025-04-16T04:04:21.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45072025-04-16T04:04:21.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
45082025-04-16T04:04:21.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45092025-04-16T04:04:21.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45102025-04-16T04:04:21.845ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45112025-04-16T04:04:21.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45122025-04-16T04:04:21.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45132025-04-16T04:04:21.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45142025-04-16T04:04:21.846ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45152025-04-16T04:04:21.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45162025-04-16T04:04:21.848ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45172025-04-16T04:04:21.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45182025-04-16T04:04:21.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45192025-04-16T04:04:21.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45202025-04-16T04:04:21.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45212025-04-16T04:04:21.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
45222025-04-16T04:04:21.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45232025-04-16T04:04:21.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45242025-04-16T04:04:21.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45252025-04-16T04:04:21.880ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
45262025-04-16T04:04:21.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45272025-04-16T04:04:21.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45282025-04-16T04:04:21.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45292025-04-16T04:04:21.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45302025-04-16T04:04:21.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45312025-04-16T04:04:21.886ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 48.15139ms iteration = 24 status = {"physical_disks_added":0}
45322025-04-16T04:04:21.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
45332025-04-16T04:04:21.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45342025-04-16T04:04:21.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45352025-04-16T04:04:21.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45362025-04-16T04:04:21.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
45372025-04-16T04:04:21.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
45382025-04-16T04:04:21.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
45392025-04-16T04:04:21.891ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
45402025-04-16T04:04:21.891ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 53.623978ms iteration = 6 status = {"blueprint_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","inventory_collection_id":"9371a2fa-681c-4714-9050-e34ced50da5a","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
45412025-04-16T04:04:21.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc99c846-5ff5-4700-bc62-acd410d495b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45422025-04-16T04:04:21.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc99c846-5ff5-4700-bc62-acd410d495b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45432025-04-16T04:04:21.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2215 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc99c846-5ff5-4700-bc62-acd410d495b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45442025-04-16T04:04:22.962ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 844839ec-8ea6-4df3-87d6-acd701a48ae7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45452025-04-16T04:04:22.962ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 844839ec-8ea6-4df3-87d6-acd701a48ae7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45462025-04-16T04:04:22.964ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2399 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 844839ec-8ea6-4df3-87d6-acd701a48ae7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45472025-04-16T04:04:23.985ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9bfc6c97-862b-4cfe-bbdf-78972cdbb847 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45482025-04-16T04:04:23.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9bfc6c97-862b-4cfe-bbdf-78972cdbb847 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45492025-04-16T04:04:23.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2587 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9bfc6c97-862b-4cfe-bbdf-78972cdbb847 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45502025-04-16T04:04:25.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4ffedba7-8bfb-44e9-9eaf-6a969fb318da roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45512025-04-16T04:04:25.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4ffedba7-8bfb-44e9-9eaf-6a969fb318da resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45522025-04-16T04:04:25.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2362 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4ffedba7-8bfb-44e9-9eaf-6a969fb318da response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45532025-04-16T04:04:26.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42cd1eea-edd0-4f55-b5d1-b416ed153a17 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45542025-04-16T04:04:26.034ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42cd1eea-edd0-4f55-b5d1-b416ed153a17 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45552025-04-16T04:04:26.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2093 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 42cd1eea-edd0-4f55-b5d1-b416ed153a17 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45562025-04-16T04:04:27.056ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b277606d-fa30-45d4-92ec-edcc48a00da6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45572025-04-16T04:04:27.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b277606d-fa30-45d4-92ec-edcc48a00da6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45582025-04-16T04:04:27.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3573 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b277606d-fa30-45d4-92ec-edcc48a00da6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45592025-04-16T04:04:28.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc079109-2aec-4a8b-ad67-ba2f170e9da5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45602025-04-16T04:04:28.091ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc079109-2aec-4a8b-ad67-ba2f170e9da5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45612025-04-16T04:04:28.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2050 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dc079109-2aec-4a8b-ad67-ba2f170e9da5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45622025-04-16T04:04:28.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 541 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 00f3fc93-8efd-4a59-866c-eea0bb30f81d response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
45632025-04-16T04:04:29.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 96271475-4b47-4ad2-816c-e3105fc2cb85 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45642025-04-16T04:04:29.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 96271475-4b47-4ad2-816c-e3105fc2cb85 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45652025-04-16T04:04:29.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2524 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 96271475-4b47-4ad2-816c-e3105fc2cb85 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45662025-04-16T04:04:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
45672025-04-16T04:04:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
45682025-04-16T04:04:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45692025-04-16T04:04:29.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45702025-04-16T04:04:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45712025-04-16T04:04:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45722025-04-16T04:04:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45732025-04-16T04:04:29.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45742025-04-16T04:04:29.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45752025-04-16T04:04:29.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.919108ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45762025-04-16T04:04:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45772025-04-16T04:04:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45782025-04-16T04:04:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45792025-04-16T04:04:29.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45802025-04-16T04:04:29.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45812025-04-16T04:04:29.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45822025-04-16T04:04:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
45832025-04-16T04:04:29.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45842025-04-16T04:04:29.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45852025-04-16T04:04:29.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
45862025-04-16T04:04:29.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.861822ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
45872025-04-16T04:04:30.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b1cf0f-2e43-4747-9384-a236afa2c022 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45882025-04-16T04:04:30.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b1cf0f-2e43-4747-9384-a236afa2c022 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45892025-04-16T04:04:30.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b1cf0f-2e43-4747-9384-a236afa2c022 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45902025-04-16T04:04:31.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2dd27707-87e6-453a-bba2-3c17bfc42016 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45912025-04-16T04:04:31.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2dd27707-87e6-453a-bba2-3c17bfc42016 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45922025-04-16T04:04:31.158ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2dd27707-87e6-453a-bba2-3c17bfc42016 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45932025-04-16T04:04:32.178ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd60c1c2-5b81-4c9b-9acb-8950500c3e99 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45942025-04-16T04:04:32.178ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd60c1c2-5b81-4c9b-9acb-8950500c3e99 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45952025-04-16T04:04:32.180ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1857 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dd60c1c2-5b81-4c9b-9acb-8950500c3e99 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45962025-04-16T04:04:33.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbc1519c-3b81-4eb0-810e-e12eef9ae291 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45972025-04-16T04:04:33.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbc1519c-3b81-4eb0-810e-e12eef9ae291 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45982025-04-16T04:04:33.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2396 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbc1519c-3b81-4eb0-810e-e12eef9ae291 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
45992025-04-16T04:04:34.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3c4dfe80-a03b-44ee-b2e4-ab88496e2a92 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46002025-04-16T04:04:34.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3c4dfe80-a03b-44ee-b2e4-ab88496e2a92 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46012025-04-16T04:04:34.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3c4dfe80-a03b-44ee-b2e4-ab88496e2a92 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46022025-04-16T04:04:35.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bdb08776-1079-4ec3-9a23-e5386e82e4a6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46032025-04-16T04:04:35.243ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bdb08776-1079-4ec3-9a23-e5386e82e4a6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46042025-04-16T04:04:35.245ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2124 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bdb08776-1079-4ec3-9a23-e5386e82e4a6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46052025-04-16T04:04:36.265ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 22bd7ca6-eb4c-4c19-9571-4c5724ab6060 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46062025-04-16T04:04:36.265ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 22bd7ca6-eb4c-4c19-9571-4c5724ab6060 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46072025-04-16T04:04:36.267ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2309 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 22bd7ca6-eb4c-4c19-9571-4c5724ab6060 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46082025-04-16T04:04:37.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2c9a4f28-fa4a-42ee-97cb-037cab0d5b77 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46092025-04-16T04:04:37.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2c9a4f28-fa4a-42ee-97cb-037cab0d5b77 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46102025-04-16T04:04:37.289ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2c9a4f28-fa4a-42ee-97cb-037cab0d5b77 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46112025-04-16T04:04:38.309ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb137b49-0e97-4580-8ee3-18c2ef6cc3ee roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46122025-04-16T04:04:38.310ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb137b49-0e97-4580-8ee3-18c2ef6cc3ee resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46132025-04-16T04:04:38.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2081 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb137b49-0e97-4580-8ee3-18c2ef6cc3ee response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46142025-04-16T04:04:38.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 482 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 3bf1a6b4-a4b9-4c30-b3a5-242a724e4ffa response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
46152025-04-16T04:04:39.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ad22dbf1-df59-449e-a9e3-497b16f19474 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46162025-04-16T04:04:39.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ad22dbf1-df59-449e-a9e3-497b16f19474 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46172025-04-16T04:04:39.335ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2443 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ad22dbf1-df59-449e-a9e3-497b16f19474 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46182025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
46192025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
46202025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46212025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46222025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46232025-04-16T04:04:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46242025-04-16T04:04:39.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46252025-04-16T04:04:39.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46262025-04-16T04:04:39.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46272025-04-16T04:04:39.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.617461ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46282025-04-16T04:04:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46292025-04-16T04:04:39.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46302025-04-16T04:04:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46312025-04-16T04:04:39.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46322025-04-16T04:04:39.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46332025-04-16T04:04:39.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46342025-04-16T04:04:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
46352025-04-16T04:04:39.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46362025-04-16T04:04:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46372025-04-16T04:04:39.464ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
46382025-04-16T04:04:39.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.377393ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
46392025-04-16T04:04:40.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 09ecf390-4db9-43f9-b3d9-559ab2a5e36c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46402025-04-16T04:04:40.356ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 09ecf390-4db9-43f9-b3d9-559ab2a5e36c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46412025-04-16T04:04:40.358ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2108 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 09ecf390-4db9-43f9-b3d9-559ab2a5e36c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46422025-04-16T04:04:41.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d3913c8c-f2ee-496b-a985-17567495a8b6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46432025-04-16T04:04:41.378ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d3913c8c-f2ee-496b-a985-17567495a8b6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46442025-04-16T04:04:41.380ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2215 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d3913c8c-f2ee-496b-a985-17567495a8b6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46452025-04-16T04:04:42.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803f779b-b40d-46ab-b555-4fb19670fd99 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46462025-04-16T04:04:42.400ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803f779b-b40d-46ab-b555-4fb19670fd99 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46472025-04-16T04:04:42.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1913 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803f779b-b40d-46ab-b555-4fb19670fd99 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46482025-04-16T04:04:43.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb83c6df-b2ad-4819-a231-1d310bf36301 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46492025-04-16T04:04:43.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb83c6df-b2ad-4819-a231-1d310bf36301 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46502025-04-16T04:04:43.424ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bb83c6df-b2ad-4819-a231-1d310bf36301 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46512025-04-16T04:04:44.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 59f5d52f-7a08-47b7-a995-33eb17db7d21 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46522025-04-16T04:04:44.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 59f5d52f-7a08-47b7-a995-33eb17db7d21 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46532025-04-16T04:04:44.446ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2016 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 59f5d52f-7a08-47b7-a995-33eb17db7d21 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46542025-04-16T04:04:45.467ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 85976f4d-e395-4956-a990-5359bb03a76c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46552025-04-16T04:04:45.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 85976f4d-e395-4956-a990-5359bb03a76c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46562025-04-16T04:04:45.469ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 85976f4d-e395-4956-a990-5359bb03a76c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46572025-04-16T04:04:46.488ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 442422cb-28b4-4ba2-b9ff-0c5a7060c695 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46582025-04-16T04:04:46.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 442422cb-28b4-4ba2-b9ff-0c5a7060c695 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46592025-04-16T04:04:46.490ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2153 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 442422cb-28b4-4ba2-b9ff-0c5a7060c695 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46602025-04-16T04:04:47.510ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6841f557-c9ad-4ae4-8075-e94e4f6f33ee roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46612025-04-16T04:04:47.510ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6841f557-c9ad-4ae4-8075-e94e4f6f33ee resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46622025-04-16T04:04:47.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2241 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6841f557-c9ad-4ae4-8075-e94e4f6f33ee response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46632025-04-16T04:04:48.532ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef15ed69-2d9e-4b6b-a7a9-ba98e1235ce6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46642025-04-16T04:04:48.532ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef15ed69-2d9e-4b6b-a7a9-ba98e1235ce6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46652025-04-16T04:04:48.534ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2579 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ef15ed69-2d9e-4b6b-a7a9-ba98e1235ce6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
46662025-04-16T04:04:48.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 453 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = cfde05cb-57eb-49db-89e5-4cddba4a5e0c response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
46672025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
46682025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
46692025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46702025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46712025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
46722025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46732025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
46742025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46752025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
46762025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46772025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
46782025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
46792025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
46802025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
46812025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46822025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
46832025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
46842025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46852025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
46862025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46872025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
46882025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46892025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
46902025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46912025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
46922025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
46932025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
46942025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
46952025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
46962025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46972025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
46982025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
46992025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47002025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
47012025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
47022025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47032025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
47042025-04-16T04:04:49.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
47052025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47062025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47072025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47082025-04-16T04:04:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47092025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47102025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47112025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47122025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
47132025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47142025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47152025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
47162025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47172025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
47182025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47192025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47202025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47212025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
47222025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47232025-04-16T04:04:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47242025-04-16T04:04:49.449ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47252025-04-16T04:04:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
47262025-04-16T04:04:49.451ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47272025-04-16T04:04:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47282025-04-16T04:04:49.451ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3179, tv_nsec: 732296574 } })
47292025-04-16T04:04:49.451ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
47302025-04-16T04:04:49.453ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47312025-04-16T04:04:49.453ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47322025-04-16T04:04:49.453ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47332025-04-16T04:04:49.454ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.661692ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
47342025-04-16T04:04:49.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47352025-04-16T04:04:49.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47362025-04-16T04:04:49.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47372025-04-16T04:04:49.455ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47382025-04-16T04:04:49.455ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
47392025-04-16T04:04:49.455ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47402025-04-16T04:04:49.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47412025-04-16T04:04:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
47422025-04-16T04:04:49.456ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47432025-04-16T04:04:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47442025-04-16T04:04:49.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47452025-04-16T04:04:49.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47462025-04-16T04:04:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47472025-04-16T04:04:49.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47482025-04-16T04:04:49.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47492025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47502025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47512025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47522025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47532025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
47542025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47552025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47562025-04-16T04:04:49.462ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47572025-04-16T04:04:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
47582025-04-16T04:04:49.463ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47592025-04-16T04:04:49.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47602025-04-16T04:04:49.464ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3179, tv_nsec: 744891622 } })
47612025-04-16T04:04:49.464ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3179, tv_nsec: 744915203 } })
47622025-04-16T04:04:49.464ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
47632025-04-16T04:04:49.464ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
47642025-04-16T04:04:49.465ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 18.22884ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
47652025-04-16T04:04:49.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47662025-04-16T04:04:49.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47672025-04-16T04:04:49.466ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47682025-04-16T04:04:49.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47692025-04-16T04:04:49.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47702025-04-16T04:04:49.466ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 19.937373ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47712025-04-16T04:04:49.468ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47722025-04-16T04:04:49.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47732025-04-16T04:04:49.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47742025-04-16T04:04:49.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47752025-04-16T04:04:49.470ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47762025-04-16T04:04:49.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47772025-04-16T04:04:49.472ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47782025-04-16T04:04:49.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
47792025-04-16T04:04:49.474ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47802025-04-16T04:04:49.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47812025-04-16T04:04:49.474ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47822025-04-16T04:04:49.474ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 27.899471ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47832025-04-16T04:04:49.489ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47842025-04-16T04:04:49.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47852025-04-16T04:04:49.490ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 43.241193ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
47862025-04-16T04:04:49.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 49.512457ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
47872025-04-16T04:04:49.497ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47882025-04-16T04:04:49.497ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47892025-04-16T04:04:49.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47902025-04-16T04:04:49.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 51.331393ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47912025-04-16T04:04:49.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47922025-04-16T04:04:49.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47932025-04-16T04:04:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47942025-04-16T04:04:49.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47952025-04-16T04:04:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
47962025-04-16T04:04:49.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47972025-04-16T04:04:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47982025-04-16T04:04:49.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47992025-04-16T04:04:49.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48002025-04-16T04:04:49.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48012025-04-16T04:04:49.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48022025-04-16T04:04:49.506ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 59.852192ms iteration = 22 status = 0
48032025-04-16T04:04:49.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48042025-04-16T04:04:49.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48052025-04-16T04:04:49.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48062025-04-16T04:04:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48072025-04-16T04:04:49.508ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
48082025-04-16T04:04:49.508ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48092025-04-16T04:04:49.508ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "47a8ebb9-f6d2-4e3c-a4b7-d2c4b3e87bba", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48102025-04-16T04:04:49.508ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48112025-04-16T04:04:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
48122025-04-16T04:04:49.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48132025-04-16T04:04:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48142025-04-16T04:04:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
48152025-04-16T04:04:49.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48162025-04-16T04:04:49.510ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
48172025-04-16T04:04:49.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48182025-04-16T04:04:49.517ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48192025-04-16T04:04:49.517ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
48202025-04-16T04:04:49.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48212025-04-16T04:04:49.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48222025-04-16T04:04:49.519ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
48232025-04-16T04:04:49.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.362908ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
48242025-04-16T04:04:49.524ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 77.705714ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48252025-04-16T04:04:49.533ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
48262025-04-16T04:04:49.533ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48272025-04-16T04:04:49.533ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bc6cfb3-1ce6-48b9-8b7c-7d623eef7177", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48282025-04-16T04:04:49.533ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48292025-04-16T04:04:49.534ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
48302025-04-16T04:04:49.534ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48312025-04-16T04:04:49.534ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff03c40-8a7e-4339-901b-120fe48f996b", "content-length": "26", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48322025-04-16T04:04:49.534ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48332025-04-16T04:04:49.536ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
48342025-04-16T04:04:49.537ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
48352025-04-16T04:04:49.537ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48362025-04-16T04:04:49.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48372025-04-16T04:04:49.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.968565ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
48382025-04-16T04:04:49.542ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48392025-04-16T04:04:49.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48402025-04-16T04:04:49.554ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
48412025-04-16T04:04:49.554ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 107.979601ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
48422025-04-16T04:04:49.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 32d6297a-e2bd-42b1-aa39-177b32ac4808 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
48432025-04-16T04:04:49.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 32d6297a-e2bd-42b1-aa39-177b32ac4808 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
48442025-04-16T04:04:49.557ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48452025-04-16T04:04:49.557ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
48462025-04-16T04:04:49.557ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 110.641703ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
48472025-04-16T04:04:49.594ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 147.995892ms iteration = 25 status = {"physical_disks_added":0}
48482025-04-16T04:04:49.613ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
48492025-04-16T04:04:49.613ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48502025-04-16T04:04:49.614ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8ac82a6-f23c-4a7c-aa58-d187889c15e2", "content-length": "372", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48512025-04-16T04:04:49.614ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48522025-04-16T04:04:49.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48532025-04-16T04:04:49.615ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48542025-04-16T04:04:49.616ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 61000 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 32d6297a-e2bd-42b1-aa39-177b32ac4808 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
48552025-04-16T04:04:49.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48562025-04-16T04:04:49.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48572025-04-16T04:04:49.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48582025-04-16T04:04:49.618ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48592025-04-16T04:04:49.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48602025-04-16T04:04:49.620ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48612025-04-16T04:04:49.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48622025-04-16T04:04:49.625ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 179.31163ms iteration = 22 status = {}
48632025-04-16T04:04:49.636ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48642025-04-16T04:04:49.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48652025-04-16T04:04:49.638ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 191.903728ms iteration = 22 status = null
48662025-04-16T04:04:49.657ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
48672025-04-16T04:04:49.657ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
48682025-04-16T04:04:49.658ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1263d793-a320-42c6-82a5-486ff268cd00", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48692025-04-16T04:04:49.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48702025-04-16T04:04:49.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48712025-04-16T04:04:49.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48722025-04-16T04:04:49.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48732025-04-16T04:04:49.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48742025-04-16T04:04:49.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48752025-04-16T04:04:49.666ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48762025-04-16T04:04:49.666ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48772025-04-16T04:04:49.667ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "69068e8e-7e01-4d78-939a-a2d93db1fd99", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
48782025-04-16T04:04:49.669ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 222.868129ms iteration = 22 status = {}
48792025-04-16T04:04:49.700ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48802025-04-16T04:04:49.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48812025-04-16T04:04:49.703ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
48822025-04-16T04:04:49.703ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48832025-04-16T04:04:49.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48842025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
48852025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
48862025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
48872025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48882025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48892025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48902025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48912025-04-16T04:04:49.718ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48922025-04-16T04:04:49.719ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48932025-04-16T04:04:49.719ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48942025-04-16T04:04:49.719ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
48952025-04-16T04:04:49.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48962025-04-16T04:04:49.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48972025-04-16T04:04:49.722ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
48982025-04-16T04:04:49.722ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48992025-04-16T04:04:49.727ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49002025-04-16T04:04:49.838ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6577c8e2-7309-45e2-82e0-ab33b941ccaf", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
49012025-04-16T04:04:49.838ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
49022025-04-16T04:04:49.838ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
49032025-04-16T04:04:49.838ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c95e2b7e-e54f-481f-be84-83a0f90e4236", "content-length": "691", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
49042025-04-16T04:04:49.839ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
49052025-04-16T04:04:49.839ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
49062025-04-16T04:04:49.839ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
49072025-04-16T04:04:49.839ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2876ffbf-4d6f-420d-afee-462263584df7", "content-length": "682", "date": "Wed, 16 Apr 2025 04:04:49 GMT"} })
49082025-04-16T04:04:49.839ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49092025-04-16T04:04:49.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49102025-04-16T04:04:49.841ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 394.497437ms iteration = 22 status = {}
49112025-04-16T04:04:49.843ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49122025-04-16T04:04:49.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49132025-04-16T04:04:49.847ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49142025-04-16T04:04:49.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49152025-04-16T04:04:49.849ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49162025-04-16T04:04:49.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49172025-04-16T04:04:49.857ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 410.566464ms iteration = 22 status = {}
49182025-04-16T04:04:50.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 74e0bddd-d27d-427a-90fd-3a22c01fba24 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49192025-04-16T04:04:50.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 74e0bddd-d27d-427a-90fd-3a22c01fba24 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49202025-04-16T04:04:50.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2461 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 74e0bddd-d27d-427a-90fd-3a22c01fba24 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49212025-04-16T04:04:51.658ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1df2263f-2a83-4503-b12a-aa6c5ab4a133 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49222025-04-16T04:04:51.659ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1df2263f-2a83-4503-b12a-aa6c5ab4a133 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49232025-04-16T04:04:51.661ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2208 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1df2263f-2a83-4503-b12a-aa6c5ab4a133 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49242025-04-16T04:04:52.680ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a971584-f1a4-4799-8af9-df0930d80f9d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49252025-04-16T04:04:52.681ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a971584-f1a4-4799-8af9-df0930d80f9d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49262025-04-16T04:04:52.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a971584-f1a4-4799-8af9-df0930d80f9d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49272025-04-16T04:04:53.702ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5513bda3-78c1-4da5-a562-672d6e4e0f61 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49282025-04-16T04:04:53.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5513bda3-78c1-4da5-a562-672d6e4e0f61 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49292025-04-16T04:04:53.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2425 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5513bda3-78c1-4da5-a562-672d6e4e0f61 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49302025-04-16T04:04:54.724ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 176834c0-5791-4139-9f4e-7390de179e0a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49312025-04-16T04:04:54.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 176834c0-5791-4139-9f4e-7390de179e0a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49322025-04-16T04:04:54.726ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2140 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 176834c0-5791-4139-9f4e-7390de179e0a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49332025-04-16T04:04:55.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9363d4e7-a9b3-42b9-8f88-7cc64880a1d9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49342025-04-16T04:04:55.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9363d4e7-a9b3-42b9-8f88-7cc64880a1d9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49352025-04-16T04:04:55.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1736 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9363d4e7-a9b3-42b9-8f88-7cc64880a1d9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49362025-04-16T04:04:56.768ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e1e022a-d28e-4f25-ad39-642891c8fbef roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49372025-04-16T04:04:56.769ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e1e022a-d28e-4f25-ad39-642891c8fbef resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49382025-04-16T04:04:56.771ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2719 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e1e022a-d28e-4f25-ad39-642891c8fbef response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49392025-04-16T04:04:57.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f620c25-b606-4800-802f-d55288cbaf19 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49402025-04-16T04:04:57.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f620c25-b606-4800-802f-d55288cbaf19 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49412025-04-16T04:04:57.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7f620c25-b606-4800-802f-d55288cbaf19 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49422025-04-16T04:04:58.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 503 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = cd897a1a-6bd2-443b-a9a0-12b336fdc1ed response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
49432025-04-16T04:04:58.813ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 768569cc-7cab-4ed4-9baa-4f5e3cc029c4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49442025-04-16T04:04:58.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 768569cc-7cab-4ed4-9baa-4f5e3cc029c4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49452025-04-16T04:04:58.815ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 768569cc-7cab-4ed4-9baa-4f5e3cc029c4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49462025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
49472025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
49482025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49492025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49502025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49512025-04-16T04:04:59.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49522025-04-16T04:04:59.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49532025-04-16T04:04:59.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49542025-04-16T04:04:59.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49552025-04-16T04:04:59.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49562025-04-16T04:04:59.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49572025-04-16T04:04:59.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49582025-04-16T04:04:59.451ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.928528ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49592025-04-16T04:04:59.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49602025-04-16T04:04:59.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49612025-04-16T04:04:59.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49622025-04-16T04:04:59.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
49632025-04-16T04:04:59.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49642025-04-16T04:04:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49652025-04-16T04:04:59.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
49662025-04-16T04:04:59.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.542985ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
49672025-04-16T04:04:59.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 923d6df3-5bd4-449e-88bb-2f7251314b19 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49682025-04-16T04:04:59.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 923d6df3-5bd4-449e-88bb-2f7251314b19 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49692025-04-16T04:04:59.837ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1890 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 923d6df3-5bd4-449e-88bb-2f7251314b19 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49702025-04-16T04:05:00.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28d54f2b-60ad-4129-ab37-37c7f72d1c63 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49712025-04-16T04:05:00.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28d54f2b-60ad-4129-ab37-37c7f72d1c63 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49722025-04-16T04:05:00.859ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1941 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 28d54f2b-60ad-4129-ab37-37c7f72d1c63 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49732025-04-16T04:05:01.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41471ee7-ca18-417b-bbb7-3d4b2a517e57 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49742025-04-16T04:05:01.878ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41471ee7-ca18-417b-bbb7-3d4b2a517e57 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49752025-04-16T04:05:01.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1942 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41471ee7-ca18-417b-bbb7-3d4b2a517e57 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49762025-04-16T04:05:02.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c5dfacc-2e35-4a16-94f6-52ebd2ddf4c9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49772025-04-16T04:05:02.900ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c5dfacc-2e35-4a16-94f6-52ebd2ddf4c9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49782025-04-16T04:05:02.902ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2340 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c5dfacc-2e35-4a16-94f6-52ebd2ddf4c9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49792025-04-16T04:05:03.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fcfba20c-6558-401a-a716-96b7b52f209f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49802025-04-16T04:05:03.928ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fcfba20c-6558-401a-a716-96b7b52f209f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49812025-04-16T04:05:03.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fcfba20c-6558-401a-a716-96b7b52f209f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49822025-04-16T04:05:04.951ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c18e4eb9-16c8-4387-beac-55889d2cb755 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49832025-04-16T04:05:04.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c18e4eb9-16c8-4387-beac-55889d2cb755 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49842025-04-16T04:05:04.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c18e4eb9-16c8-4387-beac-55889d2cb755 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49852025-04-16T04:05:05.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5047412b-b528-4470-9b0b-a32ccb18d45f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49862025-04-16T04:05:05.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5047412b-b528-4470-9b0b-a32ccb18d45f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49872025-04-16T04:05:05.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1877 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5047412b-b528-4470-9b0b-a32ccb18d45f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49882025-04-16T04:05:06.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ab749069-d1be-4d0a-a322-826477f404fc roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49892025-04-16T04:05:06.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ab749069-d1be-4d0a-a322-826477f404fc resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49902025-04-16T04:05:06.999ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2506 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ab749069-d1be-4d0a-a322-826477f404fc response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49912025-04-16T04:05:08.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c61322bc-3716-420b-b830-a2588f141772 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49922025-04-16T04:05:08.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c61322bc-3716-420b-b830-a2588f141772 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49932025-04-16T04:05:08.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2716 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c61322bc-3716-420b-b830-a2588f141772 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49942025-04-16T04:05:08.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 808 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 6cc78b5e-7596-4d13-b373-86acd815a688 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
49952025-04-16T04:05:09.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02196ef2-86b3-4ed3-9e61-0c265b1ba1d4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49962025-04-16T04:05:09.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02196ef2-86b3-4ed3-9e61-0c265b1ba1d4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49972025-04-16T04:05:09.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2191 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 02196ef2-86b3-4ed3-9e61-0c265b1ba1d4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
49982025-04-16T04:05:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
49992025-04-16T04:05:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
50002025-04-16T04:05:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50012025-04-16T04:05:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50022025-04-16T04:05:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50032025-04-16T04:05:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50042025-04-16T04:05:09.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50052025-04-16T04:05:09.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50062025-04-16T04:05:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50072025-04-16T04:05:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50082025-04-16T04:05:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50092025-04-16T04:05:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50102025-04-16T04:05:09.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50112025-04-16T04:05:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50122025-04-16T04:05:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50132025-04-16T04:05:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
50142025-04-16T04:05:09.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50152025-04-16T04:05:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50162025-04-16T04:05:09.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
50172025-04-16T04:05:09.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.758229ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
50182025-04-16T04:05:09.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.264887ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50192025-04-16T04:05:10.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ea20ab3c-0ca9-4045-a52b-9f2a16757e96 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50202025-04-16T04:05:10.064ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ea20ab3c-0ca9-4045-a52b-9f2a16757e96 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50212025-04-16T04:05:10.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ea20ab3c-0ca9-4045-a52b-9f2a16757e96 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50222025-04-16T04:05:11.087ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9aec818c-cb20-454f-be38-abc01d53e085 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50232025-04-16T04:05:11.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9aec818c-cb20-454f-be38-abc01d53e085 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50242025-04-16T04:05:11.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2963 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9aec818c-cb20-454f-be38-abc01d53e085 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50252025-04-16T04:05:12.110ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e3a3c763-944a-4023-94f9-75936ace80f1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50262025-04-16T04:05:12.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e3a3c763-944a-4023-94f9-75936ace80f1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50272025-04-16T04:05:12.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2411 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e3a3c763-944a-4023-94f9-75936ace80f1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50282025-04-16T04:05:13.137ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebc1c138-2cf3-4dfd-952e-b7e5b42f69a1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50292025-04-16T04:05:13.137ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebc1c138-2cf3-4dfd-952e-b7e5b42f69a1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50302025-04-16T04:05:13.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1968 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebc1c138-2cf3-4dfd-952e-b7e5b42f69a1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50312025-04-16T04:05:14.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d249f76a-4fea-49f8-acc9-6ad1c99223c2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50322025-04-16T04:05:14.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d249f76a-4fea-49f8-acc9-6ad1c99223c2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50332025-04-16T04:05:14.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1698 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d249f76a-4fea-49f8-acc9-6ad1c99223c2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50342025-04-16T04:05:15.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94b3e656-1864-4953-90a6-93c5a9b75b5b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50352025-04-16T04:05:15.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94b3e656-1864-4953-90a6-93c5a9b75b5b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50362025-04-16T04:05:15.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94b3e656-1864-4953-90a6-93c5a9b75b5b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50372025-04-16T04:05:16.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 71d0a13e-3617-4026-8dd4-da7886e806d7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50382025-04-16T04:05:16.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 71d0a13e-3617-4026-8dd4-da7886e806d7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50392025-04-16T04:05:16.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 71d0a13e-3617-4026-8dd4-da7886e806d7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50402025-04-16T04:05:17.228ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc3a67d5-966d-4b1e-8191-538040638b68 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50412025-04-16T04:05:17.228ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc3a67d5-966d-4b1e-8191-538040638b68 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50422025-04-16T04:05:17.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc3a67d5-966d-4b1e-8191-538040638b68 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50432025-04-16T04:05:18.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10d724c1-84bf-4d44-b38c-540c8fadad50 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50442025-04-16T04:05:18.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10d724c1-84bf-4d44-b38c-540c8fadad50 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50452025-04-16T04:05:18.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2548 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10d724c1-84bf-4d44-b38c-540c8fadad50 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50462025-04-16T04:05:18.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 559 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = e25d4a61-7f10-4fb2-89c8-f3c2eb09c86e response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
50472025-04-16T04:05:19.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ef5ea41-b4b1-4336-88d9-612f187736b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50482025-04-16T04:05:19.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ef5ea41-b4b1-4336-88d9-612f187736b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50492025-04-16T04:05:19.277ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1683 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ef5ea41-b4b1-4336-88d9-612f187736b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
50502025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
50512025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
50522025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50532025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
50542025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
50552025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
50562025-04-16T04:05:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
50572025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
50582025-04-16T04:05:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
50592025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50602025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
50612025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50622025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
50632025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
50642025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50652025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50662025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
50672025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
50682025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
50692025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
50702025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
50712025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50722025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
50732025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
50742025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
50752025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
50762025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
50772025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50782025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
50792025-04-16T04:05:19.447ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:55:19.447126965 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
50802025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
50812025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
50822025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
50832025-04-16T04:05:19.447ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
50842025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
50852025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50862025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.841\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
50872025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
50882025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
50892025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
50902025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
50912025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
50922025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50932025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50942025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50952025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
50962025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50972025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
50982025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
50992025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51002025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
51012025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
51022025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
51032025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
51042025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51052025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
51062025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
51072025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
51082025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
51092025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
51102025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
51112025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
51122025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51132025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
51142025-04-16T04:05:19.447ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
51152025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
51162025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
51172025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
51182025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51192025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51202025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51212025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51222025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51232025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
51242025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
51252025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51262025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51272025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51282025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51292025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
51302025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
51312025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
51322025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
51332025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51342025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51352025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51362025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51372025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
51382025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
51392025-04-16T04:05:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3209, tv_nsec: 728496012 } })
51402025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
51412025-04-16T04:05:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51422025-04-16T04:05:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3209, tv_nsec: 728530482 } })
51432025-04-16T04:05:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51442025-04-16T04:05:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
51452025-04-16T04:05:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.329956ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
51462025-04-16T04:05:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
51472025-04-16T04:05:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.363677ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
51482025-04-16T04:05:19.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51492025-04-16T04:05:19.451ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3209, tv_nsec: 731760506 } })
51502025-04-16T04:05:19.451ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.963958ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
51512025-04-16T04:05:19.451ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
51522025-04-16T04:05:19.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51532025-04-16T04:05:19.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51542025-04-16T04:05:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51552025-04-16T04:05:19.456ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
51562025-04-16T04:05:19.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
51572025-04-16T04:05:19.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51582025-04-16T04:05:19.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51592025-04-16T04:05:19.458ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:55:19.447126965 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
51602025-04-16T04:05:19.458ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 10.897225ms iteration = 12 status = {"expiration":"2025-04-16T03:55:19.447126965Z","pruned":[]}
51612025-04-16T04:05:19.461ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51622025-04-16T04:05:19.461ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51632025-04-16T04:05:19.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51642025-04-16T04:05:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
51652025-04-16T04:05:19.463ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51662025-04-16T04:05:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51672025-04-16T04:05:19.466ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3209, tv_nsec: 747503577 } })
51682025-04-16T04:05:19.467ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
51692025-04-16T04:05:19.493ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51702025-04-16T04:05:19.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
51712025-04-16T04:05:19.497ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51722025-04-16T04:05:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51732025-04-16T04:05:19.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 50.544207ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
51742025-04-16T04:05:19.501ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51752025-04-16T04:05:19.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51762025-04-16T04:05:19.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51772025-04-16T04:05:19.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51782025-04-16T04:05:19.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51792025-04-16T04:05:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
51802025-04-16T04:05:19.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
51812025-04-16T04:05:19.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51822025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51832025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51842025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
51852025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51862025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51872025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51882025-04-16T04:05:19.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51892025-04-16T04:05:19.508ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 61.537654ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
51902025-04-16T04:05:19.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51912025-04-16T04:05:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51922025-04-16T04:05:19.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51932025-04-16T04:05:19.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51942025-04-16T04:05:19.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
51952025-04-16T04:05:19.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51962025-04-16T04:05:19.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51972025-04-16T04:05:19.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51982025-04-16T04:05:19.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51992025-04-16T04:05:19.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52002025-04-16T04:05:19.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 64.84324ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
52012025-04-16T04:05:19.512ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52022025-04-16T04:05:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52032025-04-16T04:05:19.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52042025-04-16T04:05:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52052025-04-16T04:05:19.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52062025-04-16T04:05:19.516ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52072025-04-16T04:05:19.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52082025-04-16T04:05:19.519ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092025-04-16T04:05:19.521ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52102025-04-16T04:05:19.521ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
52112025-04-16T04:05:19.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52122025-04-16T04:05:19.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52132025-04-16T04:05:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
52142025-04-16T04:05:19.522ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52152025-04-16T04:05:19.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52162025-04-16T04:05:19.523ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52172025-04-16T04:05:19.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52182025-04-16T04:05:19.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52192025-04-16T04:05:19.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52202025-04-16T04:05:19.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52212025-04-16T04:05:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52222025-04-16T04:05:19.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52232025-04-16T04:05:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52242025-04-16T04:05:19.527ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
52252025-04-16T04:05:19.527ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
52262025-04-16T04:05:19.527ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 80.101561ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
52272025-04-16T04:05:19.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52282025-04-16T04:05:19.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52292025-04-16T04:05:19.549ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
52302025-04-16T04:05:19.549ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.241808ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
52312025-04-16T04:05:19.549ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52322025-04-16T04:05:19.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52332025-04-16T04:05:19.557ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52342025-04-16T04:05:19.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52352025-04-16T04:05:19.560ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
52362025-04-16T04:05:19.560ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
52372025-04-16T04:05:19.560ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
52382025-04-16T04:05:19.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52392025-04-16T04:05:19.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52402025-04-16T04:05:19.567ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
52412025-04-16T04:05:19.569ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
52422025-04-16T04:05:19.571ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
52432025-04-16T04:05:19.571ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
52442025-04-16T04:05:19.571ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
52452025-04-16T04:05:19.571ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52462025-04-16T04:05:19.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52472025-04-16T04:05:19.586ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
52482025-04-16T04:05:19.586ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52492025-04-16T04:05:19.586ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1d57d61-d179-4a3c-be38-49902fccd90e", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
52502025-04-16T04:05:19.586ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52512025-04-16T04:05:19.588ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
52522025-04-16T04:05:19.591ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52532025-04-16T04:05:19.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52542025-04-16T04:05:19.608ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
52552025-04-16T04:05:19.608ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52562025-04-16T04:05:19.609ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7c3a5b9-af5d-4e63-a5e0-6bafc066552b", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
52572025-04-16T04:05:19.609ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52582025-04-16T04:05:19.611ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
52592025-04-16T04:05:19.655ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
52602025-04-16T04:05:19.655ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 208.24258ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
52612025-04-16T04:05:19.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 229.687004ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
52622025-04-16T04:05:19.677ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3209, tv_nsec: 958349929 } })
52632025-04-16T04:05:19.678ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
52642025-04-16T04:05:19.678ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52652025-04-16T04:05:19.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
52662025-04-16T04:05:19.681ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52672025-04-16T04:05:19.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52682025-04-16T04:05:19.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52692025-04-16T04:05:19.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52702025-04-16T04:05:19.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52712025-04-16T04:05:19.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52722025-04-16T04:05:19.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52732025-04-16T04:05:19.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52742025-04-16T04:05:19.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52752025-04-16T04:05:19.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52762025-04-16T04:05:19.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52772025-04-16T04:05:19.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52782025-04-16T04:05:19.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52792025-04-16T04:05:19.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52802025-04-16T04:05:19.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52812025-04-16T04:05:19.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52822025-04-16T04:05:19.726ZINFOnexus (BackgroundTasks): set region 15481f07-8d2f-4feb-a43a-f1868cd7a85e port as 19000 background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:127
52832025-04-16T04:05:19.729ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52842025-04-16T04:05:19.729ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 282.18409ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52852025-04-16T04:05:19.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52862025-04-16T04:05:19.729ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
52872025-04-16T04:05:19.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
52882025-04-16T04:05:19.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52892025-04-16T04:05:19.736ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52902025-04-16T04:05:19.736ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 289.514585ms iteration = 23 status = null
52912025-04-16T04:05:19.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52922025-04-16T04:05:19.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
52932025-04-16T04:05:19.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52942025-04-16T04:05:19.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52952025-04-16T04:05:19.738ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 291.273789ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
52962025-04-16T04:05:19.740ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52972025-04-16T04:05:19.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
52982025-04-16T04:05:19.744ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52992025-04-16T04:05:19.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53002025-04-16T04:05:19.746ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 299.623384ms iteration = 23 status = 0
53012025-04-16T04:05:19.747ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53022025-04-16T04:05:19.747ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
53032025-04-16T04:05:19.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53042025-04-16T04:05:19.747ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
53052025-04-16T04:05:19.747ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
53062025-04-16T04:05:19.747ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 300.491031ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
53072025-04-16T04:05:19.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53082025-04-16T04:05:19.751ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 304.013071ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
53092025-04-16T04:05:19.751ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
53102025-04-16T04:05:19.751ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 304.846728ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
53112025-04-16T04:05:19.754ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 307.446828ms iteration = 23 status = {}
53122025-04-16T04:05:19.755ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
53132025-04-16T04:05:19.756ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
53142025-04-16T04:05:19.756ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 309.089051ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
53152025-04-16T04:05:19.756ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53162025-04-16T04:05:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53172025-04-16T04:05:19.757ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
53182025-04-16T04:05:19.757ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
53192025-04-16T04:05:19.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53202025-04-16T04:05:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53212025-04-16T04:05:19.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53222025-04-16T04:05:19.759ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53232025-04-16T04:05:19.761ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
53242025-04-16T04:05:19.762ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "162b959e-bce9-4685-aa93-d132f07a7991", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53252025-04-16T04:05:19.762ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53262025-04-16T04:05:19.762ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
53272025-04-16T04:05:19.762ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
53282025-04-16T04:05:19.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53292025-04-16T04:05:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53302025-04-16T04:05:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53312025-04-16T04:05:19.763ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53322025-04-16T04:05:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53332025-04-16T04:05:19.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53342025-04-16T04:05:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53352025-04-16T04:05:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53362025-04-16T04:05:19.764ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b9931c80-4ded-463f-aaae-700eeef24f51", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53372025-04-16T04:05:19.764ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53382025-04-16T04:05:19.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53392025-04-16T04:05:19.767ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53402025-04-16T04:05:19.767ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 319.952826ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
53412025-04-16T04:05:19.767ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
53422025-04-16T04:05:19.767ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
53432025-04-16T04:05:19.767ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
53442025-04-16T04:05:19.767ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 320.71061ms iteration = 12 status = {"generation":1}
53452025-04-16T04:05:19.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53462025-04-16T04:05:19.769ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53472025-04-16T04:05:19.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53482025-04-16T04:05:19.770ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
53492025-04-16T04:05:19.770ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
53502025-04-16T04:05:19.771ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
53512025-04-16T04:05:19.771ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 323.796291ms iteration = 12 status = {"generation":2}
53522025-04-16T04:05:19.775ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 327.975553ms iteration = 26 status = {"physical_disks_added":0}
53532025-04-16T04:05:19.795ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53542025-04-16T04:05:19.795ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
53552025-04-16T04:05:19.795ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 348.103461ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
53562025-04-16T04:05:19.796ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53572025-04-16T04:05:19.798ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
53582025-04-16T04:05:19.798ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53592025-04-16T04:05:19.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53602025-04-16T04:05:19.820ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53612025-04-16T04:05:19.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53622025-04-16T04:05:19.823ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53632025-04-16T04:05:19.826ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53642025-04-16T04:05:19.826ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53652025-04-16T04:05:19.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53662025-04-16T04:05:19.830ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53672025-04-16T04:05:19.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53682025-04-16T04:05:19.833ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53692025-04-16T04:05:19.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53702025-04-16T04:05:19.859ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
53712025-04-16T04:05:19.859ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 412.029562ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
53722025-04-16T04:05:19.892ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
53732025-04-16T04:05:19.892ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53742025-04-16T04:05:19.892ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f749c65-6c98-4751-b55e-74d682c0838c", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
53752025-04-16T04:05:19.893ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53762025-04-16T04:05:19.895ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
53772025-04-16T04:05:19.906ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
53782025-04-16T04:05:19.907ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "aa2ab6e8-cfee-44e8-b6d8-0c29f23a25d9", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53792025-04-16T04:05:19.907ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53802025-04-16T04:05:19.909ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "72ca1e8d-56ed-492b-9241-790492db13b4", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53812025-04-16T04:05:19.909ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53822025-04-16T04:05:19.912ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4bb191d3-7398-4328-9690-0cdb7504d32b", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53832025-04-16T04:05:19.912ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53842025-04-16T04:05:19.914ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
53852025-04-16T04:05:19.914ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 467.676051ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
53862025-04-16T04:05:19.919ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
53872025-04-16T04:05:19.919ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53882025-04-16T04:05:19.920ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed8c54f-f00c-40b3-9d07-7adf04edbdf8", "content-length": "372", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
53892025-04-16T04:05:19.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53902025-04-16T04:05:19.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53912025-04-16T04:05:19.929ZINFOnexus (BackgroundTasks): set region 88381701-f845-4108-872d-daa7713f8c11 port as 19002 background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:127
53922025-04-16T04:05:19.929ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 482.083545ms iteration = 12 status = {"errors":[],"found_port_ok":["set region 15481f07-8d2f-4feb-a43a-f1868cd7a85e port as 19000","set region 88381701-f845-4108-872d-daa7713f8c11 port as 19002"]}
53932025-04-16T04:05:19.987ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
53942025-04-16T04:05:19.987ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
53952025-04-16T04:05:19.987ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2b78eb5b-cfa6-4984-9a2f-d706acaceb6c", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
53962025-04-16T04:05:19.987ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53972025-04-16T04:05:19.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53982025-04-16T04:05:19.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53992025-04-16T04:05:19.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54002025-04-16T04:05:19.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54012025-04-16T04:05:19.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54022025-04-16T04:05:19.998ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
54032025-04-16T04:05:19.998ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
54042025-04-16T04:05:19.998ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ce98743d-ffba-43f3-82b7-8d4e39108e43", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
54052025-04-16T04:05:20.000ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 553.512305ms iteration = 23 status = {}
54062025-04-16T04:05:20.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54072025-04-16T04:05:20.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54082025-04-16T04:05:20.059ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
54092025-04-16T04:05:20.059ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54102025-04-16T04:05:20.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54112025-04-16T04:05:20.061ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
54122025-04-16T04:05:20.062ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b0dbf61-960a-4dc2-9ec3-96c32aff946f", "content-length": "691", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
54132025-04-16T04:05:20.062ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
54142025-04-16T04:05:20.062ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
54152025-04-16T04:05:20.062ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
54162025-04-16T04:05:20.063ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 616.372386ms iteration = 23 status = {}
54172025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
54182025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
54192025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
54202025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
54212025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54222025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54232025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54242025-04-16T04:05:20.071ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54252025-04-16T04:05:20.072ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54262025-04-16T04:05:20.072ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54272025-04-16T04:05:20.072ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
54282025-04-16T04:05:20.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54292025-04-16T04:05:20.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54302025-04-16T04:05:20.074ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
54312025-04-16T04:05:20.074ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
54322025-04-16T04:05:20.179ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8defaa39-6a12-460f-a73f-2bf8b29bc2e6", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
54332025-04-16T04:05:20.179ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
54342025-04-16T04:05:20.179ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
54352025-04-16T04:05:20.179ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "396339d1-11ec-4533-81af-5ef6b3e71997", "content-length": "682", "date": "Wed, 16 Apr 2025 04:05:19 GMT"} })
54362025-04-16T04:05:20.179ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54372025-04-16T04:05:20.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54382025-04-16T04:05:20.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54392025-04-16T04:05:20.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54402025-04-16T04:05:20.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54412025-04-16T04:05:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54422025-04-16T04:05:20.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54432025-04-16T04:05:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54442025-04-16T04:05:20.206ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 759.606123ms iteration = 23 status = {}
54452025-04-16T04:05:20.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e650ef43-05a3-4979-8a2b-141cb2acb3c7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54462025-04-16T04:05:20.299ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e650ef43-05a3-4979-8a2b-141cb2acb3c7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54472025-04-16T04:05:20.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1625 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e650ef43-05a3-4979-8a2b-141cb2acb3c7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54482025-04-16T04:05:20.642ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
54492025-04-16T04:05:20.642ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
54502025-04-16T04:05:20.673ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
54512025-04-16T04:05:20.673ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:38095
54522025-04-16T04:05:20.673ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38095 req_id = 8daf7159-9233-4648-aabf-35b0b990f9a8 roles = RoleSet { roles: {} } uri = /metrics/producers
54532025-04-16T04:05:20.674ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38095 req_id = 8daf7159-9233-4648-aabf-35b0b990f9a8 resource = Database result = Ok(()) uri = /metrics/producers
54542025-04-16T04:05:20.717ZINFOnexus: assigned collector to new producer collector_id = 0efaecbf-c4c4-456a-96b6-2cd10690be0f file = nexus/src/app/oximeter.rs:76 producer_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
54552025-04-16T04:05:20.717ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 43520 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:38095 req_id = 8daf7159-9233-4648-aabf-35b0b990f9a8 response_code = 201 uri = /metrics/producers
54562025-04-16T04:05:20.717ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "8daf7159-9233-4648-aabf-35b0b990f9a8", "content-length": "41", "date": "Wed, 16 Apr 2025 04:05:20 GMT"} })
54572025-04-16T04:05:20.718ZDEBGnexus (producer-registration-task): registered with nexus successfully
54582025-04-16T04:05:20.718ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
54592025-04-16T04:05:21.320ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f2d8ba8-1e43-4a70-b6b3-abe72821aa91 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54602025-04-16T04:05:21.321ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f2d8ba8-1e43-4a70-b6b3-abe72821aa91 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54612025-04-16T04:05:21.322ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1988 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f2d8ba8-1e43-4a70-b6b3-abe72821aa91 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54622025-04-16T04:05:22.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1b4024b2-4d15-4a22-abb5-9c2aad5c644f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54632025-04-16T04:05:22.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1b4024b2-4d15-4a22-abb5-9c2aad5c644f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54642025-04-16T04:05:22.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2386 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1b4024b2-4d15-4a22-abb5-9c2aad5c644f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54652025-04-16T04:05:23.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5c375393-b072-4c0b-9425-c84b436b1295 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54662025-04-16T04:05:23.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5c375393-b072-4c0b-9425-c84b436b1295 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54672025-04-16T04:05:23.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2409 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5c375393-b072-4c0b-9425-c84b436b1295 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54682025-04-16T04:05:24.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 29b2b251-a3fb-4f52-b002-db24a93f9382 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54692025-04-16T04:05:24.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 29b2b251-a3fb-4f52-b002-db24a93f9382 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54702025-04-16T04:05:24.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2175 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 29b2b251-a3fb-4f52-b002-db24a93f9382 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54712025-04-16T04:05:25.415ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5e6011ee-7a8d-4de2-9514-8100c44d316d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54722025-04-16T04:05:25.415ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5e6011ee-7a8d-4de2-9514-8100c44d316d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54732025-04-16T04:05:25.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3887 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5e6011ee-7a8d-4de2-9514-8100c44d316d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54742025-04-16T04:05:26.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41f333cd-d1ea-44d3-afc9-0606e198cc43 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54752025-04-16T04:05:26.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41f333cd-d1ea-44d3-afc9-0606e198cc43 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54762025-04-16T04:05:26.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3824 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 41f333cd-d1ea-44d3-afc9-0606e198cc43 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54772025-04-16T04:05:27.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3e2cfa73-efed-44f5-ab1c-4e1c93c0ac01 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54782025-04-16T04:05:27.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3e2cfa73-efed-44f5-ab1c-4e1c93c0ac01 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54792025-04-16T04:05:27.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3e2cfa73-efed-44f5-ab1c-4e1c93c0ac01 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54802025-04-16T04:05:28.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5bac54db-869b-4bcc-8541-f9f1f1613d25 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54812025-04-16T04:05:28.498ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5bac54db-869b-4bcc-8541-f9f1f1613d25 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54822025-04-16T04:05:28.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2510 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5bac54db-869b-4bcc-8541-f9f1f1613d25 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
54832025-04-16T04:05:28.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 524 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 1c1bb428-fb44-474e-9b92-c39511a38d6b response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
54842025-04-16T04:05:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
54852025-04-16T04:05:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
54862025-04-16T04:05:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54872025-04-16T04:05:29.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54882025-04-16T04:05:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54892025-04-16T04:05:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54902025-04-16T04:05:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54912025-04-16T04:05:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54922025-04-16T04:05:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54932025-04-16T04:05:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54942025-04-16T04:05:29.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54952025-04-16T04:05:29.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54962025-04-16T04:05:29.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54972025-04-16T04:05:29.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
54982025-04-16T04:05:29.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54992025-04-16T04:05:29.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55002025-04-16T04:05:29.473ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
55012025-04-16T04:05:29.473ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.87686ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
55022025-04-16T04:05:29.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55032025-04-16T04:05:29.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55042025-04-16T04:05:29.480ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.082703ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55052025-04-16T04:05:29.520ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c82e2fa8-f947-43ae-9973-dc8bd32a1d5d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55062025-04-16T04:05:29.521ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c82e2fa8-f947-43ae-9973-dc8bd32a1d5d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55072025-04-16T04:05:29.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1913 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c82e2fa8-f947-43ae-9973-dc8bd32a1d5d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55082025-04-16T04:05:30.544ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbb50008-f41b-433d-ac79-cf4650b9751b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55092025-04-16T04:05:30.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbb50008-f41b-433d-ac79-cf4650b9751b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55102025-04-16T04:05:30.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2330 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbb50008-f41b-433d-ac79-cf4650b9751b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55112025-04-16T04:05:31.327ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0b76fb6c-52df-4927-be4a-759cbff9dc93 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-04-16T04:05:31.306397326Z }
55122025-04-16T04:05:31.328ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0b76fb6c-52df-4927-be4a-759cbff9dc93 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55132025-04-16T04:05:31.328ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0b76fb6c-52df-4927-be4a-759cbff9dc93 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55142025-04-16T04:05:31.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 62643 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0b76fb6c-52df-4927-be4a-759cbff9dc93 response_code = 204 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55152025-04-16T04:05:31.401ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 848781ce-fa93-4cc7-94ec-19d95899d4b0 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-04-16T04:05:31.401048474Z }
55162025-04-16T04:05:31.401ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 848781ce-fa93-4cc7-94ec-19d95899d4b0 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55172025-04-16T04:05:31.401ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 848781ce-fa93-4cc7-94ec-19d95899d4b0 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55182025-04-16T04:05:31.410ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 8858 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 848781ce-fa93-4cc7-94ec-19d95899d4b0 response_code = 204 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55192025-04-16T04:05:31.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7af6e66f-8866-4f42-977f-a9aa56179c60 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55202025-04-16T04:05:31.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7af6e66f-8866-4f42-977f-a9aa56179c60 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55212025-04-16T04:05:31.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1786 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7af6e66f-8866-4f42-977f-a9aa56179c60 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55222025-04-16T04:05:32.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ca4974c-0066-4218-8762-3c1f2d957ff4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55232025-04-16T04:05:32.590ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ca4974c-0066-4218-8762-3c1f2d957ff4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55242025-04-16T04:05:32.593ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2ca4974c-0066-4218-8762-3c1f2d957ff4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55252025-04-16T04:05:33.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f08352c6-fcb7-4c55-802c-8eecf491da29 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55262025-04-16T04:05:33.613ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f08352c6-fcb7-4c55-802c-8eecf491da29 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55272025-04-16T04:05:33.615ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2243 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f08352c6-fcb7-4c55-802c-8eecf491da29 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55282025-04-16T04:05:34.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6cda8158-3cfe-482f-a541-c808c85a7424 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55292025-04-16T04:05:34.638ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6cda8158-3cfe-482f-a541-c808c85a7424 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55302025-04-16T04:05:34.640ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2418 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6cda8158-3cfe-482f-a541-c808c85a7424 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55312025-04-16T04:05:35.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1aacbad0-29b4-410a-bc6e-ec02e8a838a8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55322025-04-16T04:05:35.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1aacbad0-29b4-410a-bc6e-ec02e8a838a8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55332025-04-16T04:05:35.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2408 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1aacbad0-29b4-410a-bc6e-ec02e8a838a8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55342025-04-16T04:05:36.684ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 24711f1c-bc3d-4e40-b84f-a2641d683cb8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55352025-04-16T04:05:36.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 24711f1c-bc3d-4e40-b84f-a2641d683cb8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55362025-04-16T04:05:36.686ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 24711f1c-bc3d-4e40-b84f-a2641d683cb8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55372025-04-16T04:05:37.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4b51ed82-9c84-4665-83a2-ad2f585a96b8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55382025-04-16T04:05:37.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4b51ed82-9c84-4665-83a2-ad2f585a96b8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55392025-04-16T04:05:37.710ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4b51ed82-9c84-4665-83a2-ad2f585a96b8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55402025-04-16T04:05:38.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 622 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 24c6ef85-5ce9-477c-a373-161b84da448a response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
55412025-04-16T04:05:38.731ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8770331c-fcba-4626-a91b-a93a7267b61c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55422025-04-16T04:05:38.731ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8770331c-fcba-4626-a91b-a93a7267b61c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55432025-04-16T04:05:38.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8770331c-fcba-4626-a91b-a93a7267b61c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55442025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
55452025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
55462025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55472025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55482025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55492025-04-16T04:05:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55502025-04-16T04:05:39.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55512025-04-16T04:05:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55522025-04-16T04:05:39.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55532025-04-16T04:05:39.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55542025-04-16T04:05:39.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55552025-04-16T04:05:39.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55562025-04-16T04:05:39.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55572025-04-16T04:05:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
55582025-04-16T04:05:39.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55592025-04-16T04:05:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55602025-04-16T04:05:39.468ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
55612025-04-16T04:05:39.468ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.829601ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
55622025-04-16T04:05:39.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55632025-04-16T04:05:39.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55642025-04-16T04:05:39.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.88231ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55652025-04-16T04:05:39.753ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5faa6ecb-06f2-4915-a5cb-d47dab2b1f1e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55662025-04-16T04:05:39.754ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5faa6ecb-06f2-4915-a5cb-d47dab2b1f1e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55672025-04-16T04:05:39.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2348 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5faa6ecb-06f2-4915-a5cb-d47dab2b1f1e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55682025-04-16T04:05:40.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 590c00b5-d45f-48b1-b20f-f6e6e30ac3f3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55692025-04-16T04:05:40.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 590c00b5-d45f-48b1-b20f-f6e6e30ac3f3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55702025-04-16T04:05:40.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1944 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 590c00b5-d45f-48b1-b20f-f6e6e30ac3f3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55712025-04-16T04:05:41.007ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08f6588a-232e-41a1-8d53-7785d8649c6d uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-04-16T04:05:41.007585327Z }
55722025-04-16T04:05:41.007ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08f6588a-232e-41a1-8d53-7785d8649c6d roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55732025-04-16T04:05:41.008ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08f6588a-232e-41a1-8d53-7785d8649c6d resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55742025-04-16T04:05:41.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 38102 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08f6588a-232e-41a1-8d53-7785d8649c6d response_code = 204 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55752025-04-16T04:05:41.799ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f01fe573-c0cc-4771-9533-becbe5f3684b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55762025-04-16T04:05:41.799ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f01fe573-c0cc-4771-9533-becbe5f3684b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55772025-04-16T04:05:41.801ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2346 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f01fe573-c0cc-4771-9533-becbe5f3684b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55782025-04-16T04:05:42.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2239dbfd-d4ca-4c75-aa01-0b516a5373d0 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55792025-04-16T04:05:42.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2239dbfd-d4ca-4c75-aa01-0b516a5373d0 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55802025-04-16T04:05:42.827ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 6580 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2239dbfd-d4ca-4c75-aa01-0b516a5373d0 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
55812025-04-16T04:05:43.519ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2168 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-04-16T04:05:41.046231270Z }
55822025-04-16T04:05:43.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55832025-04-16T04:05:43.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55842025-04-16T04:05:43.538ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_needs_update = true
55852025-04-16T04:05:43.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55862025-04-16T04:05:43.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55872025-04-16T04:05:43.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55882025-04-16T04:05:43.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55892025-04-16T04:05:43.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55902025-04-16T04:05:43.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55912025-04-16T04:05:43.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55922025-04-16T04:05:43.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55932025-04-16T04:05:43.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55942025-04-16T04:05:43.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55952025-04-16T04:05:43.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55962025-04-16T04:05:43.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Database result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55972025-04-16T04:05:43.553ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55982025-04-16T04:05:43.556ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, lookup_type: ById(34f3b91a-eb8a-4b88-8074-a57ae3591f56) }, key: a09ebec4-7b0e-40f0-934a-06477df23520, lookup_type: ById(a09ebec4-7b0e-40f0-934a-06477df23520) }, key: 7dc968d5-64cc-432f-95e6-ba08b4c6de5f, lookup_type: ById(7dc968d5-64cc-432f-95e6-ba08b4c6de5f) } result = Ok(()) uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
55992025-04-16T04:05:43.566ZINFOnexus (dropshot_internal): starting update saga for 7dc968d5-64cc-432f-95e6-ba08b4c6de5f actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1631 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-04-16T04:05:41.046231270Z }
56002025-04-16T04:05:43.566ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 47398 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 response_code = 204 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
56012025-04-16T04:05:43.566ZDEBGnexus (dropshot_internal): preparing instance-update saga for 7dc968d5-64cc-432f-95e6-ba08b4c6de5f... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:42814 req_id = daa280d1-3450-44c3-a087-f396bd2dfd32 uri = /vmms/ad99e3d6-895c-41d9-80a0-dc8646df3e63
56022025-04-16T04:05:43.566ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update
56032025-04-16T04:05:43.568ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"7dc968d5-64cc-432f-95e6-ba08b4c6de5f","lookup_type":{"ById":"7dc968d5-64cc-432f-95e6-ba08b4c6de5f"},"parent":{"key":"a09ebec4-7b0e-40f0-934a-06477df23520","lookup_type":{"ById":"a09ebec4-7b0e-40f0-934a-06477df23520"},"parent":{"key":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","lookup_type":{"ById":"34f3b91a-eb8a-4b88-8074-a57ae3591f56"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56042025-04-16T04:05:43.568ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update
56052025-04-16T04:05:43.608ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update
56062025-04-16T04:05:43.608ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56072025-04-16T04:05:43.610ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56082025-04-16T04:05:43.610ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56092025-04-16T04:05:43.617ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56102025-04-16T04:05:43.617ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56112025-04-16T04:05:43.635ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56122025-04-16T04:05:43.635ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56132025-04-16T04:05:43.650ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56142025-04-16T04:05:43.650ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("fac5a5a1-2a6c-4388-b983-f907cce32457")) node_id = 0 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56152025-04-16T04:05:43.656ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56162025-04-16T04:05:43.656ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56172025-04-16T04:05:43.671ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update
56182025-04-16T04:05:43.673ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56192025-04-16T04:05:43.674ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56202025-04-16T04:05:43.676ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56212025-04-16T04:05:43.676ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56222025-04-16T04:05:43.678ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56232025-04-16T04:05:43.678ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56242025-04-16T04:05:43.684ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56252025-04-16T04:05:43.684ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56262025-04-16T04:05:43.686ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56272025-04-16T04:05:43.686ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56282025-04-16T04:05:43.687ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56292025-04-16T04:05:43.688ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56302025-04-16T04:05:43.689ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56312025-04-16T04:05:43.692ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, lookup_type: ById(34f3b91a-eb8a-4b88-8074-a57ae3591f56) }, key: a09ebec4-7b0e-40f0-934a-06477df23520, lookup_type: ById(a09ebec4-7b0e-40f0-934a-06477df23520) }, key: 7dc968d5-64cc-432f-95e6-ba08b4c6de5f, lookup_type: ById(7dc968d5-64cc-432f-95e6-ba08b4c6de5f) } result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56322025-04-16T04:05:43.692ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance updater_id = fac5a5a1-2a6c-4388-b983-f907cce32457
56332025-04-16T04:05:43.692ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56342025-04-16T04:05:43.692ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56352025-04-16T04:05:43.705ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance updater_id = fac5a5a1-2a6c-4388-b983-f907cce32457
56362025-04-16T04:05:43.705ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56372025-04-16T04:05:43.705ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance
56382025-04-16T04:05:43.710ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f locked_gen = Generation(Generation(2)) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = LockInstance updater_id = fac5a5a1-2a6c-4388-b983-f907cce32457
56392025-04-16T04:05:43.710ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56402025-04-16T04:05:43.710ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("fac5a5a1-2a6c-4388-b983-f907cce32457")}) node_id = 1 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56412025-04-16T04:05:43.717ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56422025-04-16T04:05:43.717ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
56432025-04-16T04:05:43.730ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56442025-04-16T04:05:43.730ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56452025-04-16T04:05:43.731ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56462025-04-16T04:05:43.732ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56472025-04-16T04:05:43.733ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56482025-04-16T04:05:43.733ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56492025-04-16T04:05:43.734ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56502025-04-16T04:05:43.737ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, lookup_type: ById(34f3b91a-eb8a-4b88-8074-a57ae3591f56) }, key: a09ebec4-7b0e-40f0-934a-06477df23520, lookup_type: ById(a09ebec4-7b0e-40f0-934a-06477df23520) }, key: 7dc968d5-64cc-432f-95e6-ba08b4c6de5f, lookup_type: ById(7dc968d5-64cc-432f-95e6-ba08b4c6de5f) } result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56512025-04-16T04:05:43.737ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56522025-04-16T04:05:43.737ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
56532025-04-16T04:05:43.746ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: ad99e3d6-895c-41d9-80a0-dc8646df3e63, time_created: 2025-04-16T04:05:22.640736Z, time_deleted: None, instance_id: 7dc968d5-64cc-432f-95e6-ba08b4c6de5f, sled_id: fe2a235d-b62d-4168-a36a-687174761bf3, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-04-16T04:05:41.046231Z, gen: Generation(Generation(7)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-04-16T04:05:20.396379Z, gen: Generation(Generation(3)), propolis_id: Some(ad99e3d6-895c-41d9-80a0-dc8646df3e63), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(ad99e3d6-895c-41d9-80a0-dc8646df3e63 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-04-16T04:05:43.746084501Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
56542025-04-16T04:05:43.751ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56552025-04-16T04:05:43.751ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"7dc968d5-64cc-432f-95e6-ba08b4c6de5f","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"ad99e3d6-895c-41d9-80a0-dc8646df3e63"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"7dc968d5-64cc-432f-95e6-ba08b4c6de5f","lookup_type":{"ById":"7dc968d5-64cc-432f-95e6-ba08b4c6de5f"},"parent":{"key":"a09ebec4-7b0e-40f0-934a-06477df23520","lookup_type":{"ById":"a09ebec4-7b0e-40f0-934a-06477df23520"},"parent":{"key":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","lookup_type":{"ById":"34f3b91a-eb8a-4b88-8074-a57ae3591f56"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"fac5a5a1-2a6c-4388-b983-f907cce32457"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"a09ebec4-7b0e-40f0-934a-06477df23520","ram_diff":1073741824},"destroy_active_vmm":"ad99e3d6-895c-41d9-80a0-dc8646df3e63","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-04-16T04:05:43.746084501Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56562025-04-16T04:05:43.751ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56572025-04-16T04:05:43.755ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56582025-04-16T04:05:43.755ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56592025-04-16T04:05:43.755ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56602025-04-16T04:05:43.755ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56612025-04-16T04:05:43.759ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56622025-04-16T04:05:43.759ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56632025-04-16T04:05:43.762ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56642025-04-16T04:05:43.762ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56652025-04-16T04:05:43.766ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56662025-04-16T04:05:43.766ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("211ecaf2-af40-45cc-9bc9-757be71345a8")) node_id = 0 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56672025-04-16T04:05:43.770ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56682025-04-16T04:05:43.770ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56692025-04-16T04:05:43.774ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f parent_lock = UpdaterLock { updater_id: fac5a5a1-2a6c-4388-b983-f907cce32457, locked_gen: Generation(Generation(2)) } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56702025-04-16T04:05:43.774ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = BecomeUpdater
56712025-04-16T04:05:43.774ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = BecomeUpdater
56722025-04-16T04:05:43.784ZDEBGnexus (SagaExecutor): inherited lock from fac5a5a1-2a6c-4388-b983-f907cce32457 to 211ecaf2-af40-45cc-9bc9-757be71345a8 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = fac5a5a1-2a6c-4388-b983-f907cce32457 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = BecomeUpdater updater_id = 211ecaf2-af40-45cc-9bc9-757be71345a8
56732025-04-16T04:05:43.784ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56742025-04-16T04:05:43.784ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56752025-04-16T04:05:43.784ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("211ecaf2-af40-45cc-9bc9-757be71345a8")}) node_id = 1 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56762025-04-16T04:05:43.788ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56772025-04-16T04:05:43.788ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56782025-04-16T04:05:43.791ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56792025-04-16T04:05:43.791ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56802025-04-16T04:05:43.794ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
56812025-04-16T04:05:43.794ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
56822025-04-16T04:05:43.797ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
56832025-04-16T04:05:43.797ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f
56842025-04-16T04:05:43.797ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56852025-04-16T04:05:43.797ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56862025-04-16T04:05:43.807ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56872025-04-16T04:05:43.807ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56882025-04-16T04:05:43.808ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
56892025-04-16T04:05:43.808ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56902025-04-16T04:05:43.809ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UpdateNetworkConfig
56912025-04-16T04:05:43.811ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
56922025-04-16T04:05:43.811ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
56932025-04-16T04:05:43.811ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
56942025-04-16T04:05:43.812ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
56952025-04-16T04:05:43.814ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
56962025-04-16T04:05:43.814ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
56972025-04-16T04:05:43.814ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
56982025-04-16T04:05:43.821ZDEBGnexus: notifying dendrite of updates instance_id = Some(7dc968d5-64cc-432f-95e6-ba08b4c6de5f (instance)) switch = switch0
56992025-04-16T04:05:43.821ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3234, tv_nsec: 102562260 } })
57002025-04-16T04:05:43.822ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1086
57012025-04-16T04:05:43.857ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
57022025-04-16T04:05:43.857ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57032025-04-16T04:05:43.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 56215c33-b428-48ff-9379-448527f4a657 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
57042025-04-16T04:05:43.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 56215c33-b428-48ff-9379-448527f4a657 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
57052025-04-16T04:05:43.858ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "70239c5d-6468-460a-837a-da7407b57a50", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:43 GMT"} })
57062025-04-16T04:05:43.858ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57072025-04-16T04:05:43.858ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
57082025-04-16T04:05:43.859ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1815 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 56215c33-b428-48ff-9379-448527f4a657 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
57092025-04-16T04:05:43.894ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57102025-04-16T04:05:43.894ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57112025-04-16T04:05:43.898ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57122025-04-16T04:05:43.898ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57132025-04-16T04:05:43.906ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ReleaseVirtualProvisioning
57142025-04-16T04:05:43.906ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ReleaseVirtualProvisioning
57152025-04-16T04:05:43.920ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-04-16T04:05:43.907774Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, time_modified: Some(2025-04-16T04:05:43.907774Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: a09ebec4-7b0e-40f0-934a-06477df23520, time_modified: Some(2025-04-16T04:05:43.907774Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57162025-04-16T04:05:43.920ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57172025-04-16T04:05:43.920ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57182025-04-16T04:05:43.926ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57192025-04-16T04:05:43.926ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57202025-04-16T04:05:43.940ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57212025-04-16T04:05:43.940ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UnassignOximeterProducer
57222025-04-16T04:05:43.941ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UnassignOximeterProducer
57232025-04-16T04:05:43.947ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = 0efaecbf-c4c4-456a-96b6-2cd10690be0f producer_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57242025-04-16T04:05:43.947ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UnassignOximeterProducer
57252025-04-16T04:05:43.947ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = UnassignOximeterProducer
57262025-04-16T04:05:43.985ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57272025-04-16T04:05:43.985ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/7dc968d5-64cc-432f-95e6-ba08b4c6de5f
57282025-04-16T04:05:43.998ZDEBGnexus (SagaExecutor): client response oximeter-collector = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/7dc968d5-64cc-432f-95e6-ba08b4c6de5f", status: 204, headers: {"x-request-id": "b0fae9ee-e850-4c65-8507-db04c89d2225", "date": "Wed, 16 Apr 2025 04:05:43 GMT"} }) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57292025-04-16T04:05:43.998ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 0efaecbf-c4c4-456a-96b6-2cd10690be0f producer_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57302025-04-16T04:05:43.999ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57312025-04-16T04:05:43.999ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57322025-04-16T04:05:44.002ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57332025-04-16T04:05:44.003ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57342025-04-16T04:05:44.006ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f lock = UpdaterLock { updater_id: 211ecaf2-af40-45cc-9bc9-757be71345a8, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-04-16T04:05:43.746084501Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57352025-04-16T04:05:44.006ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = CommitInstanceUpdates
57362025-04-16T04:05:44.007ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = CommitInstanceUpdates
57372025-04-16T04:05:44.052ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f new_runtime = InstanceRuntimeState { time_updated: 2025-04-16T04:05:43.746084501Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57382025-04-16T04:05:44.052ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57392025-04-16T04:05:44.052ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57402025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
57412025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
57422025-04-16T04:05:44.052ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57432025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57442025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57452025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57462025-04-16T04:05:44.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57472025-04-16T04:05:44.055ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57482025-04-16T04:05:44.056ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57492025-04-16T04:05:44.056ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57502025-04-16T04:05:44.056ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57512025-04-16T04:05:44.056ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57522025-04-16T04:05:44.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57532025-04-16T04:05:44.058ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57542025-04-16T04:05:44.059ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57552025-04-16T04:05:44.059ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("7dc968d5-64cc-432f-95e6-ba08b4c6de5f"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("ad99e3d6-895c-41d9-80a0-dc8646df3e63")}) node_id = 7 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57562025-04-16T04:05:44.060ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57572025-04-16T04:05:44.060ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57582025-04-16T04:05:44.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57592025-04-16T04:05:44.063ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57602025-04-16T04:05:44.063ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57612025-04-16T04:05:44.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57622025-04-16T04:05:44.067ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57632025-04-16T04:05:44.067ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57642025-04-16T04:05:44.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57652025-04-16T04:05:44.070ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57662025-04-16T04:05:44.070ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57672025-04-16T04:05:44.073ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57682025-04-16T04:05:44.073ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ReleaseSledResources
57692025-04-16T04:05:44.073ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ReleaseSledResources
57702025-04-16T04:05:44.078ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57712025-04-16T04:05:44.078ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57722025-04-16T04:05:44.081ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57732025-04-16T04:05:44.081ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57742025-04-16T04:05:44.085ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f propolis_id = ad99e3d6-895c-41d9-80a0-dc8646df3e63 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
57752025-04-16T04:05:44.085ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = MarkVmmDeleted
57762025-04-16T04:05:44.085ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = MarkVmmDeleted
57772025-04-16T04:05:44.116ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57782025-04-16T04:05:44.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57792025-04-16T04:05:44.118ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57802025-04-16T04:05:44.119ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57812025-04-16T04:05:44.120ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57822025-04-16T04:05:44.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57832025-04-16T04:05:44.121ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57842025-04-16T04:05:44.123ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57852025-04-16T04:05:44.123ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57862025-04-16T04:05:44.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57872025-04-16T04:05:44.133ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57882025-04-16T04:05:44.133ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57892025-04-16T04:05:44.138ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57902025-04-16T04:05:44.138ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57912025-04-16T04:05:44.142ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57922025-04-16T04:05:44.142ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57932025-04-16T04:05:44.145ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
57942025-04-16T04:05:44.145ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
57952025-04-16T04:05:44.151ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
57962025-04-16T04:05:44.151ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57972025-04-16T04:05:44.152ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e586d397-e25e-4932-8c64-96951327fe86", "content-length": "372", "date": "Wed, 16 Apr 2025 04:05:43 GMT"} })
57982025-04-16T04:05:44.152ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57992025-04-16T04:05:44.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58002025-04-16T04:05:44.156ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58012025-04-16T04:05:44.156ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58022025-04-16T04:05:44.158ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58032025-04-16T04:05:44.158ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58042025-04-16T04:05:44.159ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58052025-04-16T04:05:44.160ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58062025-04-16T04:05:44.161ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58072025-04-16T04:05:44.166ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, lookup_type: ById(34f3b91a-eb8a-4b88-8074-a57ae3591f56) }, key: a09ebec4-7b0e-40f0-934a-06477df23520, lookup_type: ById(a09ebec4-7b0e-40f0-934a-06477df23520) }, key: 7dc968d5-64cc-432f-95e6-ba08b4c6de5f, lookup_type: ById(7dc968d5-64cc-432f-95e6-ba08b4c6de5f) } result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58082025-04-16T04:05:44.167ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58092025-04-16T04:05:44.167ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_node = ChainSuccessorSaga
58102025-04-16T04:05:44.172ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f karmic_state = WillReincarnate needs_reincarnation = false saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update
58112025-04-16T04:05:44.172ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58122025-04-16T04:05:44.172ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
58132025-04-16T04:05:44.176ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58142025-04-16T04:05:44.176ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
58152025-04-16T04:05:44.180ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58162025-04-16T04:05:44.180ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
58172025-04-16T04:05:44.183ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58182025-04-16T04:05:44.183ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc
58192025-04-16T04:05:44.203ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
58202025-04-16T04:05:44.203ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
58212025-04-16T04:05:44.203ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b1c3670c-1ae4-4a50-98df-2a6a5a178ba0", "date": "Wed, 16 Apr 2025 04:05:43 GMT"} })
58222025-04-16T04:05:44.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58232025-04-16T04:05:44.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58242025-04-16T04:05:44.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58252025-04-16T04:05:44.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58262025-04-16T04:05:44.222ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58272025-04-16T04:05:44.222ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc saga_name = instance-update saga_result = SagaResult { saga_id: df992fa1-53bc-44d3-a40f-78c5c51535cc, saga_log: SagaLog { saga_id: df992fa1-53bc-44d3-a40f-78c5c51535cc, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("211ecaf2-af40-45cc-9bc9-757be71345a8")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("211ecaf2-af40-45cc-9bc9-757be71345a8")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("7dc968d5-64cc-432f-95e6-ba08b4c6de5f"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("ad99e3d6-895c-41d9-80a0-dc8646df3e63")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("7dc968d5-64cc-432f-95e6-ba08b4c6de5f"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("ad99e3d6-895c-41d9-80a0-dc8646df3e63")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("211ecaf2-af40-45cc-9bc9-757be71345a8"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("211ecaf2-af40-45cc-9bc9-757be71345a8")}} }) }
58282025-04-16T04:05:44.222ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = df992fa1-53bc-44d3-a40f-78c5c51535cc instance_id = 7dc968d5-64cc-432f-95e6-ba08b4c6de5f saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update
58292025-04-16T04:05:44.222ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58302025-04-16T04:05:44.222ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
58312025-04-16T04:05:44.224ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
58322025-04-16T04:05:44.224ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0274c00f-e9fa-4270-84a5-f9547f7f883b", "content-length": "691", "date": "Wed, 16 Apr 2025 04:05:44 GMT"} })
58332025-04-16T04:05:44.224ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
58342025-04-16T04:05:44.224ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
58352025-04-16T04:05:44.224ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
58362025-04-16T04:05:44.227ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58372025-04-16T04:05:44.227ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 174.78594ms iteration = 24 status = {}
58382025-04-16T04:05:44.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58392025-04-16T04:05:44.228ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58402025-04-16T04:05:44.228ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
58412025-04-16T04:05:44.230ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
58422025-04-16T04:05:44.230ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
58432025-04-16T04:05:44.230ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d8a797a0-906b-4933-accf-407719c997f3", "date": "Wed, 16 Apr 2025 04:05:44 GMT"} })
58442025-04-16T04:05:44.232ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58452025-04-16T04:05:44.232ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
58462025-04-16T04:05:44.232ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 180.448272ms iteration = 24 status = {}
58472025-04-16T04:05:44.235ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58482025-04-16T04:05:44.235ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2
58492025-04-16T04:05:44.270ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update sec_id = 9fa72b7d-ab32-4040-b078-1b91700ee410
58502025-04-16T04:05:44.270ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = fe275d3a-a47a-4d00-b0ab-fd9ca75362f2 saga_name = start-instance-update saga_result = SagaResult { saga_id: fe275d3a-a47a-4d00-b0ab-fd9ca75362f2, saga_log: SagaLog { saga_id: fe275d3a-a47a-4d00-b0ab-fd9ca75362f2, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("fac5a5a1-2a6c-4388-b983-f907cce32457")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("fac5a5a1-2a6c-4388-b983-f907cce32457")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("fac5a5a1-2a6c-4388-b983-f907cce32457"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("fac5a5a1-2a6c-4388-b983-f907cce32457")}} }) }
58512025-04-16T04:05:44.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 26143389-f435-46c0-928d-b30dc63b60bb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58522025-04-16T04:05:44.883ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 26143389-f435-46c0-928d-b30dc63b60bb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58532025-04-16T04:05:44.885ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2669 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 26143389-f435-46c0-928d-b30dc63b60bb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58542025-04-16T04:05:45.904ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 04a2ad7f-fe73-4a4e-8945-2b639030d7dc roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58552025-04-16T04:05:45.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 04a2ad7f-fe73-4a4e-8945-2b639030d7dc resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58562025-04-16T04:05:45.906ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2002 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 04a2ad7f-fe73-4a4e-8945-2b639030d7dc response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58572025-04-16T04:05:46.928ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f73dafaa-c930-4af4-bcc5-05f8603614d9 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58582025-04-16T04:05:46.928ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f73dafaa-c930-4af4-bcc5-05f8603614d9 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58592025-04-16T04:05:46.930ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2164 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f73dafaa-c930-4af4-bcc5-05f8603614d9 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58602025-04-16T04:05:47.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = be32fc20-43d3-42b9-b7b7-57c7416f17f3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58612025-04-16T04:05:47.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = be32fc20-43d3-42b9-b7b7-57c7416f17f3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58622025-04-16T04:05:47.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2589 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = be32fc20-43d3-42b9-b7b7-57c7416f17f3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58632025-04-16T04:05:48.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 595 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = ec79c321-1f45-40c1-ae09-bebd4b2dc36b response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
58642025-04-16T04:05:48.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 047dd34c-f8f1-4b14-8a89-03b0fc8d0e4f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58652025-04-16T04:05:48.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 047dd34c-f8f1-4b14-8a89-03b0fc8d0e4f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58662025-04-16T04:05:48.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 047dd34c-f8f1-4b14-8a89-03b0fc8d0e4f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
58672025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
58682025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58692025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
58702025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58712025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
58722025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
58732025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
58742025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58752025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
58762025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
58772025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58782025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
58792025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
58802025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
58812025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
58822025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58832025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58842025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58852025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
58862025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58872025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58882025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
58892025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
58902025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58912025-04-16T04:05:49.446ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58922025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
58932025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
58942025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
58952025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58962025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58972025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
58982025-04-16T04:05:49.447ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
58992025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
59002025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
59012025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
59022025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59032025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
59042025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59052025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59062025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
59072025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
59082025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
59092025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
59102025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59112025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59122025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59132025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
59142025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
59152025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59162025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59172025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59182025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59192025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59202025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59212025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59222025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59232025-04-16T04:05:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59242025-04-16T04:05:49.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59252025-04-16T04:05:49.450ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3239, tv_nsec: 730966823 } })
59262025-04-16T04:05:49.450ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59272025-04-16T04:05:49.450ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
59282025-04-16T04:05:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
59292025-04-16T04:05:49.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59302025-04-16T04:05:49.451ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.737934ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59312025-04-16T04:05:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59322025-04-16T04:05:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
59332025-04-16T04:05:49.452ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59342025-04-16T04:05:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59352025-04-16T04:05:49.454ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3239, tv_nsec: 734937142 } })
59362025-04-16T04:05:49.454ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59372025-04-16T04:05:49.454ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
59382025-04-16T04:05:49.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.61452ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
59392025-04-16T04:05:49.456ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59402025-04-16T04:05:49.456ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
59412025-04-16T04:05:49.456ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59422025-04-16T04:05:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59432025-04-16T04:05:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
59442025-04-16T04:05:49.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59452025-04-16T04:05:49.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59462025-04-16T04:05:49.458ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59472025-04-16T04:05:49.459ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482025-04-16T04:05:49.459ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59492025-04-16T04:05:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59502025-04-16T04:05:49.461ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59512025-04-16T04:05:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59522025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
59532025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59542025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59552025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59562025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59572025-04-16T04:05:49.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59582025-04-16T04:05:49.466ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59592025-04-16T04:05:49.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59602025-04-16T04:05:49.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59612025-04-16T04:05:49.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59622025-04-16T04:05:49.471ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59632025-04-16T04:05:49.471ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
59642025-04-16T04:05:49.471ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59652025-04-16T04:05:49.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59662025-04-16T04:05:49.473ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59672025-04-16T04:05:49.474ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59682025-04-16T04:05:49.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59692025-04-16T04:05:49.474ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3239, tv_nsec: 755507418 } })
59702025-04-16T04:05:49.474ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59712025-04-16T04:05:49.475ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
59722025-04-16T04:05:49.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59732025-04-16T04:05:49.476ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59742025-04-16T04:05:49.477ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59752025-04-16T04:05:49.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59762025-04-16T04:05:49.498ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.450846ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
59772025-04-16T04:05:49.499ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 52.523126ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
59782025-04-16T04:05:49.499ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59792025-04-16T04:05:49.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59802025-04-16T04:05:49.502ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 55.596228ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59812025-04-16T04:05:49.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59822025-04-16T04:05:49.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59832025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59842025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59852025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
59862025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59872025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59882025-04-16T04:05:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59892025-04-16T04:05:49.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59902025-04-16T04:05:49.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59912025-04-16T04:05:49.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 61.225868ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
59922025-04-16T04:05:49.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59932025-04-16T04:05:49.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59942025-04-16T04:05:49.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 62.32682ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
59952025-04-16T04:05:49.511ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
59962025-04-16T04:05:49.511ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.911452ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
59972025-04-16T04:05:49.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59982025-04-16T04:05:49.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
59992025-04-16T04:05:49.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60002025-04-16T04:05:49.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60012025-04-16T04:05:49.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60022025-04-16T04:05:49.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60032025-04-16T04:05:49.517ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60042025-04-16T04:05:49.517ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
60052025-04-16T04:05:49.517ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 70.251346ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
60062025-04-16T04:05:49.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60072025-04-16T04:05:49.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
60082025-04-16T04:05:49.523ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60092025-04-16T04:05:49.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60102025-04-16T04:05:49.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
60112025-04-16T04:05:49.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.765635ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
60122025-04-16T04:05:49.532ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
60132025-04-16T04:05:49.532ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60142025-04-16T04:05:49.532ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ba9df32-a3be-4491-8b0e-ffd68c4323be", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60152025-04-16T04:05:49.532ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60162025-04-16T04:05:49.534ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60172025-04-16T04:05:49.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60182025-04-16T04:05:49.539ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60192025-04-16T04:05:49.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60202025-04-16T04:05:49.541ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
60212025-04-16T04:05:49.541ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60222025-04-16T04:05:49.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
60232025-04-16T04:05:49.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60242025-04-16T04:05:49.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60252025-04-16T04:05:49.541ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2264247-7d48-4026-b1e8-12d5dcdf3b06", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60262025-04-16T04:05:49.541ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60272025-04-16T04:05:49.543ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60282025-04-16T04:05:49.543ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.704738ms iteration = 24 status = 0
60292025-04-16T04:05:49.551ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60302025-04-16T04:05:49.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60312025-04-16T04:05:49.553ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60322025-04-16T04:05:49.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60332025-04-16T04:05:49.571ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 124.168751ms iteration = 27 status = {"physical_disks_added":0}
60342025-04-16T04:05:49.585ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
60352025-04-16T04:05:49.585ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60362025-04-16T04:05:49.585ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "064a1d84-189d-4e1d-99d6-62f3938f23ba", "content-length": "26", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60372025-04-16T04:05:49.585ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60382025-04-16T04:05:49.588ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
60392025-04-16T04:05:49.593ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 146.40333ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
60402025-04-16T04:05:49.602ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60412025-04-16T04:05:49.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60422025-04-16T04:05:49.604ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
60432025-04-16T04:05:49.604ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 157.798384ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
60442025-04-16T04:05:49.652ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
60452025-04-16T04:05:49.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
60462025-04-16T04:05:49.654ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 208.405684ms iteration = 24 status = null
60472025-04-16T04:05:49.659ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 212.677328ms iteration = 24 status = {}
60482025-04-16T04:05:49.663ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
60492025-04-16T04:05:49.663ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60502025-04-16T04:05:49.664ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fefd5195-a408-44c7-bcb4-4a28545b828f", "content-length": "372", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60512025-04-16T04:05:49.664ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60522025-04-16T04:05:49.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60532025-04-16T04:05:49.677ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
60542025-04-16T04:05:49.677ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
60552025-04-16T04:05:49.678ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "04c61fbd-a0f8-4b32-b47d-4b6bd48d3afa", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60562025-04-16T04:05:49.678ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60572025-04-16T04:05:49.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60582025-04-16T04:05:49.680ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60592025-04-16T04:05:49.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60602025-04-16T04:05:49.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60612025-04-16T04:05:49.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60622025-04-16T04:05:49.686ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
60632025-04-16T04:05:49.686ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60642025-04-16T04:05:49.687ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bc5d4514-00c1-468e-afcf-a5edf5489c37", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60652025-04-16T04:05:49.689ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 242.19181ms iteration = 25 status = {}
60662025-04-16T04:05:49.703ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60672025-04-16T04:05:49.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60682025-04-16T04:05:49.704ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60692025-04-16T04:05:49.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60702025-04-16T04:05:49.707ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60712025-04-16T04:05:49.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60722025-04-16T04:05:49.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60732025-04-16T04:05:49.711ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60742025-04-16T04:05:49.711ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60752025-04-16T04:05:49.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60762025-04-16T04:05:49.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60772025-04-16T04:05:49.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60782025-04-16T04:05:49.739ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
60792025-04-16T04:05:49.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60802025-04-16T04:05:49.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60812025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
60822025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
60832025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
60842025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
60852025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60862025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60872025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60882025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60892025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
60902025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
60912025-04-16T04:05:49.750ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
60922025-04-16T04:05:49.750ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60932025-04-16T04:05:49.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60942025-04-16T04:05:49.753ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
60952025-04-16T04:05:49.753ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
60962025-04-16T04:05:49.810ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
60972025-04-16T04:05:49.891ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3400a4ca-19af-4609-bfba-1acb533c1588", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
60982025-04-16T04:05:49.891ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
60992025-04-16T04:05:49.891ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
61002025-04-16T04:05:49.896ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6347e7f-5ade-4205-ac04-178a15d0523d", "content-length": "682", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
61012025-04-16T04:05:49.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61022025-04-16T04:05:49.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61032025-04-16T04:05:49.897ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "19754d7c-2ddb-4922-b445-90009a6e6567", "content-length": "691", "date": "Wed, 16 Apr 2025 04:05:49 GMT"} })
61042025-04-16T04:05:49.897ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
61052025-04-16T04:05:49.897ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
61062025-04-16T04:05:49.897ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
61072025-04-16T04:05:49.899ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 452.704245ms iteration = 25 status = {}
61082025-04-16T04:05:49.900ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61092025-04-16T04:05:49.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61102025-04-16T04:05:49.904ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61112025-04-16T04:05:49.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61122025-04-16T04:05:49.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61132025-04-16T04:05:49.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61142025-04-16T04:05:49.914ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 467.251312ms iteration = 24 status = {}
61152025-04-16T04:05:49.994ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b1034112-1802-496e-a481-62f660192a28 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61162025-04-16T04:05:49.994ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b1034112-1802-496e-a481-62f660192a28 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61172025-04-16T04:05:49.996ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2686 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b1034112-1802-496e-a481-62f660192a28 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61182025-04-16T04:05:51.016ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2a6e814e-746d-4e79-adf6-0819a7e08c00 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61192025-04-16T04:05:51.017ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2a6e814e-746d-4e79-adf6-0819a7e08c00 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61202025-04-16T04:05:51.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2849 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2a6e814e-746d-4e79-adf6-0819a7e08c00 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61212025-04-16T04:05:52.040ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c1901d1-ae06-4c69-bca0-4081d1efa7d7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61222025-04-16T04:05:52.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c1901d1-ae06-4c69-bca0-4081d1efa7d7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61232025-04-16T04:05:52.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2773 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9c1901d1-ae06-4c69-bca0-4081d1efa7d7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61242025-04-16T04:05:53.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0359c9e-ec14-4b7d-8e83-cd7aed697e95 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61252025-04-16T04:05:53.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0359c9e-ec14-4b7d-8e83-cd7aed697e95 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61262025-04-16T04:05:53.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c0359c9e-ec14-4b7d-8e83-cd7aed697e95 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61272025-04-16T04:05:54.086ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd5d815f-6742-49c9-a831-a2b14e58c247 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61282025-04-16T04:05:54.086ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd5d815f-6742-49c9-a831-a2b14e58c247 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61292025-04-16T04:05:54.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2649 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd5d815f-6742-49c9-a831-a2b14e58c247 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61302025-04-16T04:05:55.107ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 20763149-a6b0-49da-af24-2609f6cbf68b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61312025-04-16T04:05:55.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 20763149-a6b0-49da-af24-2609f6cbf68b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61322025-04-16T04:05:55.109ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 20763149-a6b0-49da-af24-2609f6cbf68b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61332025-04-16T04:05:56.129ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 628c41a6-69fa-43a7-86db-1993cf2d6a0c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61342025-04-16T04:05:56.130ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 628c41a6-69fa-43a7-86db-1993cf2d6a0c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61352025-04-16T04:05:56.132ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2754 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 628c41a6-69fa-43a7-86db-1993cf2d6a0c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61362025-04-16T04:05:57.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f38a58c-fe81-4771-885a-514c64d5fb7b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61372025-04-16T04:05:57.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f38a58c-fe81-4771-885a-514c64d5fb7b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61382025-04-16T04:05:57.155ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2268 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3f38a58c-fe81-4771-885a-514c64d5fb7b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61392025-04-16T04:05:58.175ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77448189-af9e-4e0c-acb2-0ec1c6d67b1a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61402025-04-16T04:05:58.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77448189-af9e-4e0c-acb2-0ec1c6d67b1a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61412025-04-16T04:05:58.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2665 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 77448189-af9e-4e0c-acb2-0ec1c6d67b1a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61422025-04-16T04:05:58.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 494 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = a2474950-0f63-4140-a436-a95c3515a320 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
61432025-04-16T04:05:59.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f270764-3ddd-4bf7-b012-b56e79fbb54a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61442025-04-16T04:05:59.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f270764-3ddd-4bf7-b012-b56e79fbb54a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61452025-04-16T04:05:59.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1978 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f270764-3ddd-4bf7-b012-b56e79fbb54a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61462025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
61472025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
61482025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61492025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61502025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61512025-04-16T04:05:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61522025-04-16T04:05:59.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61532025-04-16T04:05:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61542025-04-16T04:05:59.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61552025-04-16T04:05:59.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61562025-04-16T04:05:59.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61572025-04-16T04:05:59.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61582025-04-16T04:05:59.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61592025-04-16T04:05:59.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
61602025-04-16T04:05:59.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61612025-04-16T04:05:59.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61622025-04-16T04:05:59.465ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
61632025-04-16T04:05:59.465ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.714869ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
61642025-04-16T04:05:59.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61652025-04-16T04:05:59.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61662025-04-16T04:05:59.506ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.064956ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61672025-04-16T04:06:00.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1f5ef0ce-4331-43c8-bb26-9faa4edf4dbe roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61682025-04-16T04:06:00.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1f5ef0ce-4331-43c8-bb26-9faa4edf4dbe resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61692025-04-16T04:06:00.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2341 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1f5ef0ce-4331-43c8-bb26-9faa4edf4dbe response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61702025-04-16T04:06:01.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebe99f57-ff67-4d65-9a29-669dab434eb5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61712025-04-16T04:06:01.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebe99f57-ff67-4d65-9a29-669dab434eb5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61722025-04-16T04:06:01.244ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2389 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ebe99f57-ff67-4d65-9a29-669dab434eb5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61732025-04-16T04:06:02.264ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0949e2aa-39b5-4653-a4e0-0fd884eb3e9b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61742025-04-16T04:06:02.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0949e2aa-39b5-4653-a4e0-0fd884eb3e9b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61752025-04-16T04:06:02.266ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0949e2aa-39b5-4653-a4e0-0fd884eb3e9b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61762025-04-16T04:06:03.289ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0505728-8875-43b6-addf-c3599d8c947a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61772025-04-16T04:06:03.289ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0505728-8875-43b6-addf-c3599d8c947a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61782025-04-16T04:06:03.290ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1819 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0505728-8875-43b6-addf-c3599d8c947a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61792025-04-16T04:06:04.310ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c7dda9eb-edcd-43e2-a6dd-5b4dad9832b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61802025-04-16T04:06:04.311ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c7dda9eb-edcd-43e2-a6dd-5b4dad9832b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61812025-04-16T04:06:04.313ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2870 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c7dda9eb-edcd-43e2-a6dd-5b4dad9832b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61822025-04-16T04:06:05.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 529415e0-5c3d-451a-b882-1595da835471 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61832025-04-16T04:06:05.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 529415e0-5c3d-451a-b882-1595da835471 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61842025-04-16T04:06:05.336ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2936 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 529415e0-5c3d-451a-b882-1595da835471 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61852025-04-16T04:06:06.355ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d50cf21b-ec11-40bd-bc8a-8d3e81ce5214 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61862025-04-16T04:06:06.356ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d50cf21b-ec11-40bd-bc8a-8d3e81ce5214 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61872025-04-16T04:06:06.358ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d50cf21b-ec11-40bd-bc8a-8d3e81ce5214 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61882025-04-16T04:06:07.377ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 488ec86b-3738-491e-af77-57a20018b88d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61892025-04-16T04:06:07.378ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 488ec86b-3738-491e-af77-57a20018b88d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61902025-04-16T04:06:07.380ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2454 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 488ec86b-3738-491e-af77-57a20018b88d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61912025-04-16T04:06:08.399ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38a50e1d-b73a-46ff-81a2-0e71feaeb12c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61922025-04-16T04:06:08.399ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38a50e1d-b73a-46ff-81a2-0e71feaeb12c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61932025-04-16T04:06:08.401ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1993 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 38a50e1d-b73a-46ff-81a2-0e71feaeb12c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61942025-04-16T04:06:08.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 626 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 098a520c-2e16-4bd0-881d-a849921b3c13 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
61952025-04-16T04:06:09.420ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5934162a-a75c-4202-8ded-28e52553cf74 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61962025-04-16T04:06:09.420ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5934162a-a75c-4202-8ded-28e52553cf74 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61972025-04-16T04:06:09.422ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5934162a-a75c-4202-8ded-28e52553cf74 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
61982025-04-16T04:06:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
61992025-04-16T04:06:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62002025-04-16T04:06:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
62012025-04-16T04:06:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62022025-04-16T04:06:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62032025-04-16T04:06:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62042025-04-16T04:06:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62052025-04-16T04:06:09.449ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62062025-04-16T04:06:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62072025-04-16T04:06:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62082025-04-16T04:06:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62092025-04-16T04:06:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62102025-04-16T04:06:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62112025-04-16T04:06:09.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62122025-04-16T04:06:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62132025-04-16T04:06:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
62142025-04-16T04:06:09.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62152025-04-16T04:06:09.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62162025-04-16T04:06:09.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
62172025-04-16T04:06:09.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.447253ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
62182025-04-16T04:06:09.480ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.514621ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62192025-04-16T04:06:10.441ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fa0588e7-156c-4462-9574-7b07ecff4c0c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62202025-04-16T04:06:10.441ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fa0588e7-156c-4462-9574-7b07ecff4c0c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62212025-04-16T04:06:10.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2525 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fa0588e7-156c-4462-9574-7b07ecff4c0c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62222025-04-16T04:06:11.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5d7ad3b9-795f-4379-afa2-044f2859a8ef roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62232025-04-16T04:06:11.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5d7ad3b9-795f-4379-afa2-044f2859a8ef resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62242025-04-16T04:06:11.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5d7ad3b9-795f-4379-afa2-044f2859a8ef response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62252025-04-16T04:06:12.486ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1cbc1786-491c-4ec6-a612-88f3af1d8ac7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62262025-04-16T04:06:12.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1cbc1786-491c-4ec6-a612-88f3af1d8ac7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62272025-04-16T04:06:12.488ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2096 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1cbc1786-491c-4ec6-a612-88f3af1d8ac7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62282025-04-16T04:06:13.509ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 939d5e4e-57c2-4bbc-92e3-96e3997b55ae roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62292025-04-16T04:06:13.510ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 939d5e4e-57c2-4bbc-92e3-96e3997b55ae resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62302025-04-16T04:06:13.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2691 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 939d5e4e-57c2-4bbc-92e3-96e3997b55ae response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62312025-04-16T04:06:13.557ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:60491
62322025-04-16T04:06:13.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = e0be7e22-39b9-4c35-8ab0-9ae1a5799456 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
62332025-04-16T04:06:13.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = e0be7e22-39b9-4c35-8ab0-9ae1a5799456 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
62342025-04-16T04:06:13.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2289 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = e0be7e22-39b9-4c35-8ab0-9ae1a5799456 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
62352025-04-16T04:06:13.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 4def196d-4434-4f68-9f16-6dc57d0a4e33 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
62362025-04-16T04:06:13.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 4def196d-4434-4f68-9f16-6dc57d0a4e33 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
62372025-04-16T04:06:13.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 4def196d-4434-4f68-9f16-6dc57d0a4e33 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
62382025-04-16T04:06:14.533ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e503e603-df9a-44ad-9158-746797e56554 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62392025-04-16T04:06:14.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e503e603-df9a-44ad-9158-746797e56554 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62402025-04-16T04:06:14.536ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2501 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e503e603-df9a-44ad-9158-746797e56554 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62412025-04-16T04:06:15.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d12112d9-fb36-4abe-9299-de7cc49b9c86 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62422025-04-16T04:06:15.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d12112d9-fb36-4abe-9299-de7cc49b9c86 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62432025-04-16T04:06:15.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1760 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d12112d9-fb36-4abe-9299-de7cc49b9c86 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62442025-04-16T04:06:16.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39ab42f6-854e-477b-a442-f1710cd4c463 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62452025-04-16T04:06:16.578ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39ab42f6-854e-477b-a442-f1710cd4c463 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62462025-04-16T04:06:16.580ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2222 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 39ab42f6-854e-477b-a442-f1710cd4c463 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62472025-04-16T04:06:17.599ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d78cc961-de47-4880-b28e-d96ee0c81e33 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62482025-04-16T04:06:17.599ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d78cc961-de47-4880-b28e-d96ee0c81e33 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62492025-04-16T04:06:17.601ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2370 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d78cc961-de47-4880-b28e-d96ee0c81e33 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62502025-04-16T04:06:18.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 500 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = f39c54dc-c079-45ab-b17a-f2fe5d9235ea response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
62512025-04-16T04:06:18.621ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2223434f-ac08-42e3-984a-aa84afe5acf7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62522025-04-16T04:06:18.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2223434f-ac08-42e3-984a-aa84afe5acf7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62532025-04-16T04:06:18.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2250 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2223434f-ac08-42e3-984a-aa84afe5acf7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
62542025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
62552025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
62562025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
62572025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
62582025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
62592025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
62602025-04-16T04:06:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
62612025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62622025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
62632025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
62642025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62652025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
62662025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
62672025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
62682025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
62692025-04-16T04:06:19.446ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
62702025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62712025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.99\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
62722025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
62732025-04-16T04:06:19.446ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
62742025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62752025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
62762025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
62772025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
62782025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
62792025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
62802025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
62812025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62822025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62832025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
62842025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62852025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
62862025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
62872025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62882025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
62892025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
62902025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62912025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
62922025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
62932025-04-16T04:06:19.447ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:56:19.447107589 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
62942025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
62952025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
62962025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
62972025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
62982025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
62992025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
63002025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
63012025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
63022025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
63032025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
63042025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
63052025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
63062025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
63072025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
63082025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63092025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63102025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
63112025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
63122025-04-16T04:06:19.447ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
63132025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
63142025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
63152025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
63162025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63172025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
63182025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63192025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
63202025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
63212025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63222025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63232025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
63242025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63252025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63262025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63272025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63282025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63292025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63302025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63312025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63322025-04-16T04:06:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63332025-04-16T04:06:19.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3269, tv_nsec: 728415263 } })
63342025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63352025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63362025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63372025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
63382025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
63392025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63402025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
63412025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63422025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63432025-04-16T04:06:19.448ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3269, tv_nsec: 728581306 } })
63442025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
63452025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63462025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63472025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63482025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
63492025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63502025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63512025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
63522025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63532025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63542025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63552025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63562025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63572025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63582025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
63592025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.272185ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
63602025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
63612025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.53054ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
63622025-04-16T04:06:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63632025-04-16T04:06:19.449ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3269, tv_nsec: 730104247 } })
63642025-04-16T04:06:19.449ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
63652025-04-16T04:06:19.450ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63662025-04-16T04:06:19.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63672025-04-16T04:06:19.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63682025-04-16T04:06:19.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63692025-04-16T04:06:19.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63702025-04-16T04:06:19.451ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63712025-04-16T04:06:19.452ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.966018ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
63722025-04-16T04:06:19.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63732025-04-16T04:06:19.452ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63742025-04-16T04:06:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63752025-04-16T04:06:19.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63762025-04-16T04:06:19.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63772025-04-16T04:06:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63782025-04-16T04:06:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63792025-04-16T04:06:19.454ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63802025-04-16T04:06:19.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63812025-04-16T04:06:19.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63822025-04-16T04:06:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63832025-04-16T04:06:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63842025-04-16T04:06:19.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63852025-04-16T04:06:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63862025-04-16T04:06:19.456ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63872025-04-16T04:06:19.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63882025-04-16T04:06:19.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
63892025-04-16T04:06:19.458ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63902025-04-16T04:06:19.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63912025-04-16T04:06:19.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
63922025-04-16T04:06:19.459ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63932025-04-16T04:06:19.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63942025-04-16T04:06:19.460ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:56:19.447107589 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
63952025-04-16T04:06:19.460ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 13.055388ms iteration = 13 status = {"expiration":"2025-04-16T03:56:19.447107589Z","pruned":[]}
63962025-04-16T04:06:19.460ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63972025-04-16T04:06:19.461ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63982025-04-16T04:06:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
63992025-04-16T04:06:19.463ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64002025-04-16T04:06:19.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64012025-04-16T04:06:19.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64022025-04-16T04:06:19.464ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64032025-04-16T04:06:19.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
64042025-04-16T04:06:19.466ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3269, tv_nsec: 747316506 } })
64052025-04-16T04:06:19.466ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
64062025-04-16T04:06:19.467ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64072025-04-16T04:06:19.468ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.662578ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
64082025-04-16T04:06:19.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
64092025-04-16T04:06:19.470ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64102025-04-16T04:06:19.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64112025-04-16T04:06:19.471ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64122025-04-16T04:06:19.472ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64132025-04-16T04:06:19.473ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
64142025-04-16T04:06:19.473ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64152025-04-16T04:06:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64162025-04-16T04:06:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64172025-04-16T04:06:19.474ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64182025-04-16T04:06:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64192025-04-16T04:06:19.475ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64202025-04-16T04:06:19.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64212025-04-16T04:06:19.476ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64222025-04-16T04:06:19.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64232025-04-16T04:06:19.482ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.425529ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64242025-04-16T04:06:19.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64252025-04-16T04:06:19.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64262025-04-16T04:06:19.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64272025-04-16T04:06:19.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64282025-04-16T04:06:19.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64292025-04-16T04:06:19.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64302025-04-16T04:06:19.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64312025-04-16T04:06:19.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
64322025-04-16T04:06:19.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64332025-04-16T04:06:19.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64342025-04-16T04:06:19.491ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 43.948447ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
64352025-04-16T04:06:19.491ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3269, tv_nsec: 772300849 } })
64362025-04-16T04:06:19.491ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
64372025-04-16T04:06:19.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64382025-04-16T04:06:19.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
64392025-04-16T04:06:19.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64402025-04-16T04:06:19.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64412025-04-16T04:06:19.500ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
64422025-04-16T04:06:19.500ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
64432025-04-16T04:06:19.500ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 53.351623ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
64442025-04-16T04:06:19.500ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 53.649029ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
64452025-04-16T04:06:19.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64462025-04-16T04:06:19.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64472025-04-16T04:06:19.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64482025-04-16T04:06:19.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64492025-04-16T04:06:19.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
64502025-04-16T04:06:19.504ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64512025-04-16T04:06:19.504ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
64522025-04-16T04:06:19.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64532025-04-16T04:06:19.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64542025-04-16T04:06:19.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
64552025-04-16T04:06:19.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64562025-04-16T04:06:19.505ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64572025-04-16T04:06:19.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64582025-04-16T04:06:19.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64592025-04-16T04:06:19.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64602025-04-16T04:06:19.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
64612025-04-16T04:06:19.508ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
64622025-04-16T04:06:19.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
64632025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64642025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64652025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
64662025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64672025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64682025-04-16T04:06:19.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64692025-04-16T04:06:19.519ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64702025-04-16T04:06:19.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64712025-04-16T04:06:19.519ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
64722025-04-16T04:06:19.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 72.931009ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
64732025-04-16T04:06:19.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64742025-04-16T04:06:19.521ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64752025-04-16T04:06:19.521ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.495581ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
64762025-04-16T04:06:19.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64772025-04-16T04:06:19.523ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64782025-04-16T04:06:19.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64792025-04-16T04:06:19.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
64802025-04-16T04:06:19.527ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
64812025-04-16T04:06:19.527ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
64822025-04-16T04:06:19.527ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
64832025-04-16T04:06:19.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64842025-04-16T04:06:19.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64852025-04-16T04:06:19.530ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
64862025-04-16T04:06:19.530ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
64872025-04-16T04:06:19.530ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
64882025-04-16T04:06:19.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
64892025-04-16T04:06:19.531ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 84.479428ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
64902025-04-16T04:06:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
64912025-04-16T04:06:19.531ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64922025-04-16T04:06:19.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64932025-04-16T04:06:19.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64942025-04-16T04:06:19.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64952025-04-16T04:06:19.546ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
64962025-04-16T04:06:19.546ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 99.175338ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
64972025-04-16T04:06:19.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64982025-04-16T04:06:19.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
64992025-04-16T04:06:19.552ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65002025-04-16T04:06:19.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65012025-04-16T04:06:19.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
65022025-04-16T04:06:19.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
65032025-04-16T04:06:19.579ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
65042025-04-16T04:06:19.595ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
65052025-04-16T04:06:19.595ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65062025-04-16T04:06:19.596ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc849903-dfdd-46c3-a45f-7a3ecd9a3319", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
65072025-04-16T04:06:19.596ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65082025-04-16T04:06:19.597ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
65092025-04-16T04:06:19.614ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
65102025-04-16T04:06:19.614ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65112025-04-16T04:06:19.614ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe1a753b-ec80-4465-a213-010e712edc73", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
65122025-04-16T04:06:19.614ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65132025-04-16T04:06:19.616ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
65142025-04-16T04:06:19.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 511c2f83-4da4-4fc0-85f7-cb3ca3a5229d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
65152025-04-16T04:06:19.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 511c2f83-4da4-4fc0-85f7-cb3ca3a5229d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
65162025-04-16T04:06:19.661ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
65172025-04-16T04:06:19.688ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
65182025-04-16T04:06:19.688ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65192025-04-16T04:06:19.689ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ec34dee-830e-431a-8f1b-bdc5353ab155", "content-length": "372", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
65202025-04-16T04:06:19.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65212025-04-16T04:06:19.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65222025-04-16T04:06:19.692ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65232025-04-16T04:06:19.692ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
65242025-04-16T04:06:19.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65252025-04-16T04:06:19.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
65262025-04-16T04:06:19.695ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 248.081337ms iteration = 6 status = {"nsuccess":5}
65272025-04-16T04:06:19.695ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65282025-04-16T04:06:19.697ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65292025-04-16T04:06:19.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65302025-04-16T04:06:19.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
65312025-04-16T04:06:19.698ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65322025-04-16T04:06:19.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65332025-04-16T04:06:19.711ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 68093 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 511c2f83-4da4-4fc0-85f7-cb3ca3a5229d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
65342025-04-16T04:06:19.728ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 280.908585ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65352025-04-16T04:06:19.728ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65362025-04-16T04:06:19.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65372025-04-16T04:06:19.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65382025-04-16T04:06:19.750ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
65392025-04-16T04:06:19.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
65402025-04-16T04:06:19.751ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65412025-04-16T04:06:19.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
65422025-04-16T04:06:19.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65432025-04-16T04:06:19.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65442025-04-16T04:06:19.752ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
65452025-04-16T04:06:19.752ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
65462025-04-16T04:06:19.752ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65472025-04-16T04:06:19.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65482025-04-16T04:06:19.753ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
65492025-04-16T04:06:19.753ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
65502025-04-16T04:06:19.753ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 306.181053ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
65512025-04-16T04:06:19.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65522025-04-16T04:06:19.754ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65532025-04-16T04:06:19.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65542025-04-16T04:06:19.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65552025-04-16T04:06:19.756ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
65562025-04-16T04:06:19.756ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
65572025-04-16T04:06:19.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65582025-04-16T04:06:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65592025-04-16T04:06:19.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65602025-04-16T04:06:19.760ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65612025-04-16T04:06:19.760ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
65622025-04-16T04:06:19.760ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 313.478897ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
65632025-04-16T04:06:19.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
65642025-04-16T04:06:19.761ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65652025-04-16T04:06:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65662025-04-16T04:06:19.762ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
65672025-04-16T04:06:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
65682025-04-16T04:06:19.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65692025-04-16T04:06:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65702025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
65712025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
65722025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
65732025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 317.779653ms iteration = 13 status = {"generation":2}
65742025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
65752025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
65762025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
65772025-04-16T04:06:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 317.980666ms iteration = 13 status = {"generation":1}
65782025-04-16T04:06:19.767ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 320.628929ms iteration = 25 status = {}
65792025-04-16T04:06:19.771ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
65802025-04-16T04:06:19.771ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
65812025-04-16T04:06:19.773ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fe2bf1ee-85ed-40de-a1f8-18a2440d42b5", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
65822025-04-16T04:06:19.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65832025-04-16T04:06:19.773ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65842025-04-16T04:06:19.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65852025-04-16T04:06:19.775ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 327.884492ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
65862025-04-16T04:06:19.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
65872025-04-16T04:06:19.776ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65882025-04-16T04:06:19.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65892025-04-16T04:06:19.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65902025-04-16T04:06:19.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65912025-04-16T04:06:19.780ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 332.683257ms iteration = 25 status = 0
65922025-04-16T04:06:19.782ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65932025-04-16T04:06:19.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65942025-04-16T04:06:19.783ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
65952025-04-16T04:06:19.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
65962025-04-16T04:06:19.785ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 338.585573ms iteration = 25 status = null
65972025-04-16T04:06:19.785ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
65982025-04-16T04:06:19.786ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65992025-04-16T04:06:19.786ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "90b64b93-f897-4c36-82fb-78e162e02958", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
66002025-04-16T04:06:19.788ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 341.232306ms iteration = 26 status = {}
66012025-04-16T04:06:19.789ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66022025-04-16T04:06:19.791ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
66032025-04-16T04:06:19.791ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
66042025-04-16T04:06:19.792ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
66052025-04-16T04:06:19.792ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 344.719185ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
66062025-04-16T04:06:19.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
66072025-04-16T04:06:19.810ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
66082025-04-16T04:06:19.810ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66092025-04-16T04:06:19.810ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "07c62874-6a78-4631-a751-bf40053fdba5", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
66102025-04-16T04:06:19.810ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66112025-04-16T04:06:19.812ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
66122025-04-16T04:06:19.844ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66132025-04-16T04:06:19.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66142025-04-16T04:06:19.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66152025-04-16T04:06:19.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66162025-04-16T04:06:19.852ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66172025-04-16T04:06:19.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66182025-04-16T04:06:19.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66192025-04-16T04:06:19.856ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66202025-04-16T04:06:19.856ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66212025-04-16T04:06:19.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66222025-04-16T04:06:19.858ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
66232025-04-16T04:06:19.858ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 411.070703ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
66242025-04-16T04:06:19.860ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
66252025-04-16T04:06:19.861ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7ec24d37-521b-43e4-a652-fa7051195035", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66262025-04-16T04:06:19.861ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66272025-04-16T04:06:19.861ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
66282025-04-16T04:06:19.861ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 414.433299ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
66292025-04-16T04:06:19.863ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b02bf58e-c6b6-4054-bfe5-3092ab643162", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66302025-04-16T04:06:19.863ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66312025-04-16T04:06:19.865ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66322025-04-16T04:06:19.865ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 418.528051ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
66332025-04-16T04:06:19.873ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
66342025-04-16T04:06:19.874ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "45305854-7cf2-431d-ad02-1fd95e2152e9", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66352025-04-16T04:06:19.874ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66362025-04-16T04:06:19.875ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 428.457986ms iteration = 28 status = {"physical_disks_added":0}
66372025-04-16T04:06:19.876ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "d713bc33-dcd9-4d6b-81b4-7127a316af60", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66382025-04-16T04:06:19.876ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66392025-04-16T04:06:19.878ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ebe1177f-53b6-4602-b6b4-5909c69e2218", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66402025-04-16T04:06:19.878ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66412025-04-16T04:06:19.880ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
66422025-04-16T04:06:19.880ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 434.494776ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
66432025-04-16T04:06:20.067ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
66442025-04-16T04:06:20.067ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "704c4189-c639-4f0c-9497-37c88c03dc2b", "content-length": "691", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
66452025-04-16T04:06:20.067ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
66462025-04-16T04:06:20.067ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
66472025-04-16T04:06:20.067ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
66482025-04-16T04:06:20.068ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 622.050538ms iteration = 26 status = {}
66492025-04-16T04:06:20.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66502025-04-16T04:06:20.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66512025-04-16T04:06:20.072ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
66522025-04-16T04:06:20.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66532025-04-16T04:06:20.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66542025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
66552025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
66562025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
66572025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
66582025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66592025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66602025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66612025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66622025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66632025-04-16T04:06:20.082ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66642025-04-16T04:06:20.083ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
66652025-04-16T04:06:20.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66662025-04-16T04:06:20.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66672025-04-16T04:06:20.085ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
66682025-04-16T04:06:20.085ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
66692025-04-16T04:06:20.180ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "038ab83b-94fa-4fb9-a0c1-491f05afa033", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
66702025-04-16T04:06:20.180ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
66712025-04-16T04:06:20.180ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
66722025-04-16T04:06:20.180ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a69aa6-718d-49f7-bca4-d8edaef67036", "content-length": "682", "date": "Wed, 16 Apr 2025 04:06:19 GMT"} })
66732025-04-16T04:06:20.180ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66742025-04-16T04:06:20.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66752025-04-16T04:06:20.184ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66762025-04-16T04:06:20.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66772025-04-16T04:06:20.187ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66782025-04-16T04:06:20.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66792025-04-16T04:06:20.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66802025-04-16T04:06:20.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66812025-04-16T04:06:20.198ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 751.653316ms iteration = 25 status = {}
66822025-04-16T04:06:20.738ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ce3d6e4-3f62-4839-b92d-c00284d59132 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66832025-04-16T04:06:20.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ce3d6e4-3f62-4839-b92d-c00284d59132 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66842025-04-16T04:06:20.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2491 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ce3d6e4-3f62-4839-b92d-c00284d59132 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66852025-04-16T04:06:21.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 62ac5d32-361d-4fc8-b284-005e94963de0 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66862025-04-16T04:06:21.759ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 62ac5d32-361d-4fc8-b284-005e94963de0 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66872025-04-16T04:06:21.761ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2508 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 62ac5d32-361d-4fc8-b284-005e94963de0 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66882025-04-16T04:06:22.780ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bee0ed65-aebd-4e8f-b4e4-9706c6cda51f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66892025-04-16T04:06:22.780ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bee0ed65-aebd-4e8f-b4e4-9706c6cda51f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66902025-04-16T04:06:22.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2826 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bee0ed65-aebd-4e8f-b4e4-9706c6cda51f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66912025-04-16T04:06:23.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd0ac44c-559e-4784-a96c-c2f0e234bca2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66922025-04-16T04:06:23.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd0ac44c-559e-4784-a96c-c2f0e234bca2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66932025-04-16T04:06:23.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2786 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fd0ac44c-559e-4784-a96c-c2f0e234bca2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66942025-04-16T04:06:24.826ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08586efd-6004-4ea7-a461-6ae9fb533268 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66952025-04-16T04:06:24.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08586efd-6004-4ea7-a461-6ae9fb533268 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66962025-04-16T04:06:24.829ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2538 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 08586efd-6004-4ea7-a461-6ae9fb533268 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66972025-04-16T04:06:25.848ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0e0906f9-0e82-433e-8d93-d5bf891bcb8a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66982025-04-16T04:06:25.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0e0906f9-0e82-433e-8d93-d5bf891bcb8a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
66992025-04-16T04:06:25.851ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2700 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0e0906f9-0e82-433e-8d93-d5bf891bcb8a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67002025-04-16T04:06:26.872ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd1437e5-acc4-4f4d-a8a6-2570d316cff5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67012025-04-16T04:06:26.872ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd1437e5-acc4-4f4d-a8a6-2570d316cff5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67022025-04-16T04:06:26.874ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bd1437e5-acc4-4f4d-a8a6-2570d316cff5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67032025-04-16T04:06:27.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f5c5f2c-0b95-40f3-a5d3-9cf7e1dde7fe roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67042025-04-16T04:06:27.894ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f5c5f2c-0b95-40f3-a5d3-9cf7e1dde7fe resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67052025-04-16T04:06:27.896ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4f5c5f2c-0b95-40f3-a5d3-9cf7e1dde7fe response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67062025-04-16T04:06:28.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 518 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 7adce229-8533-4e43-98c0-be5cdf1c76e4 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
67072025-04-16T04:06:28.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7b28544e-84c2-43d2-bdcc-721301a6aa34 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67082025-04-16T04:06:28.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7b28544e-84c2-43d2-bdcc-721301a6aa34 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67092025-04-16T04:06:28.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1819 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7b28544e-84c2-43d2-bdcc-721301a6aa34 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67102025-04-16T04:06:29.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
67112025-04-16T04:06:29.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67122025-04-16T04:06:29.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
67132025-04-16T04:06:29.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67142025-04-16T04:06:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67152025-04-16T04:06:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67162025-04-16T04:06:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67172025-04-16T04:06:29.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67182025-04-16T04:06:29.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67192025-04-16T04:06:29.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67202025-04-16T04:06:29.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67212025-04-16T04:06:29.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67222025-04-16T04:06:29.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67232025-04-16T04:06:29.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67242025-04-16T04:06:29.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67252025-04-16T04:06:29.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
67262025-04-16T04:06:29.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67272025-04-16T04:06:29.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67282025-04-16T04:06:29.495ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
67292025-04-16T04:06:29.495ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.886045ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
67302025-04-16T04:06:29.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.413102ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67312025-04-16T04:06:29.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 329eef36-b107-4281-a0cf-b8f2bd65b860 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67322025-04-16T04:06:29.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 329eef36-b107-4281-a0cf-b8f2bd65b860 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67332025-04-16T04:06:29.939ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1935 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 329eef36-b107-4281-a0cf-b8f2bd65b860 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67342025-04-16T04:06:30.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0bd86c4-997b-4db4-80c1-7fa475db70ff roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67352025-04-16T04:06:30.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0bd86c4-997b-4db4-80c1-7fa475db70ff resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67362025-04-16T04:06:30.962ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f0bd86c4-997b-4db4-80c1-7fa475db70ff response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67372025-04-16T04:06:31.981ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b3dd06-8cf9-490e-af50-f3ab9ac3689e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67382025-04-16T04:06:31.981ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b3dd06-8cf9-490e-af50-f3ab9ac3689e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67392025-04-16T04:06:31.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 31b3dd06-8cf9-490e-af50-f3ab9ac3689e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67402025-04-16T04:06:33.003ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6fa80bf8-9825-4f90-951f-b051adb09e28 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67412025-04-16T04:06:33.003ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6fa80bf8-9825-4f90-951f-b051adb09e28 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67422025-04-16T04:06:33.005ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2132 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6fa80bf8-9825-4f90-951f-b051adb09e28 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67432025-04-16T04:06:34.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1dbba45e-9ac4-47e9-af8e-39c31d5e9eeb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67442025-04-16T04:06:34.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1dbba45e-9ac4-47e9-af8e-39c31d5e9eeb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67452025-04-16T04:06:34.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2642 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1dbba45e-9ac4-47e9-af8e-39c31d5e9eeb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67462025-04-16T04:06:35.047ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 565b6ddf-7067-483a-9673-fea07cd7fb8a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67472025-04-16T04:06:35.048ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 565b6ddf-7067-483a-9673-fea07cd7fb8a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67482025-04-16T04:06:35.050ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2631 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 565b6ddf-7067-483a-9673-fea07cd7fb8a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67492025-04-16T04:06:36.069ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 73dbd420-366a-4f14-b009-0699fb3a2662 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67502025-04-16T04:06:36.070ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 73dbd420-366a-4f14-b009-0699fb3a2662 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67512025-04-16T04:06:36.071ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2126 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 73dbd420-366a-4f14-b009-0699fb3a2662 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67522025-04-16T04:06:37.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49102cd4-8590-4472-a7af-9c0cbc86cdd5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67532025-04-16T04:06:37.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49102cd4-8590-4472-a7af-9c0cbc86cdd5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67542025-04-16T04:06:37.094ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2194 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 49102cd4-8590-4472-a7af-9c0cbc86cdd5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67552025-04-16T04:06:38.114ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 366ea31c-b69f-43d4-8a7b-cdf58572a2b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67562025-04-16T04:06:38.114ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 366ea31c-b69f-43d4-8a7b-cdf58572a2b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67572025-04-16T04:06:38.116ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2249 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 366ea31c-b69f-43d4-8a7b-cdf58572a2b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67582025-04-16T04:06:38.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 518 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 44b28b92-a68b-4143-8677-44dbdc944752 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
67592025-04-16T04:06:39.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff7c240a-85bc-4240-ac64-415b29b94d74 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67602025-04-16T04:06:39.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff7c240a-85bc-4240-ac64-415b29b94d74 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67612025-04-16T04:06:39.138ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2770 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ff7c240a-85bc-4240-ac64-415b29b94d74 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67622025-04-16T04:06:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
67632025-04-16T04:06:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
67642025-04-16T04:06:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67652025-04-16T04:06:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67662025-04-16T04:06:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67672025-04-16T04:06:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67682025-04-16T04:06:39.448ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67692025-04-16T04:06:39.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67702025-04-16T04:06:39.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67712025-04-16T04:06:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67722025-04-16T04:06:39.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67732025-04-16T04:06:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67742025-04-16T04:06:39.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67752025-04-16T04:06:39.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67762025-04-16T04:06:39.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67772025-04-16T04:06:39.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
67782025-04-16T04:06:39.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67792025-04-16T04:06:39.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67802025-04-16T04:06:39.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
67812025-04-16T04:06:39.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.844801ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
67822025-04-16T04:06:39.478ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.567734ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67832025-04-16T04:06:40.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65682b89-e529-4cc9-a4e4-8e6d4d4db684 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67842025-04-16T04:06:40.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65682b89-e529-4cc9-a4e4-8e6d4d4db684 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67852025-04-16T04:06:40.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2549 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 65682b89-e529-4cc9-a4e4-8e6d4d4db684 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67862025-04-16T04:06:41.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18487a48-3bb8-4f95-9710-f9c7abb7c75a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67872025-04-16T04:06:41.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18487a48-3bb8-4f95-9710-f9c7abb7c75a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67882025-04-16T04:06:41.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18487a48-3bb8-4f95-9710-f9c7abb7c75a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67892025-04-16T04:06:42.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c48c88cc-6faa-4c24-ab92-37236b7cea6a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67902025-04-16T04:06:42.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c48c88cc-6faa-4c24-ab92-37236b7cea6a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67912025-04-16T04:06:42.205ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2159 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c48c88cc-6faa-4c24-ab92-37236b7cea6a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67922025-04-16T04:06:43.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5641bfba-6603-4b7b-bc3a-f18bef8a708c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67932025-04-16T04:06:43.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5641bfba-6603-4b7b-bc3a-f18bef8a708c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67942025-04-16T04:06:43.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2654 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5641bfba-6603-4b7b-bc3a-f18bef8a708c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
67952025-04-16T04:06:43.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 834d023a-9f65-4037-a8c8-5c8d1b3a315d roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
67962025-04-16T04:06:43.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 834d023a-9f65-4037-a8c8-5c8d1b3a315d resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
67972025-04-16T04:06:43.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2854 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 834d023a-9f65-4037-a8c8-5c8d1b3a315d response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
67982025-04-16T04:06:43.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = de01bbb9-2b6e-4550-8103-9362cbded558 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
67992025-04-16T04:06:43.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = de01bbb9-2b6e-4550-8103-9362cbded558 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
68002025-04-16T04:06:43.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = de01bbb9-2b6e-4550-8103-9362cbded558 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
68012025-04-16T04:06:44.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45fe3d4d-c271-4785-8d3b-d5bb998bbe9b roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68022025-04-16T04:06:44.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45fe3d4d-c271-4785-8d3b-d5bb998bbe9b resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68032025-04-16T04:06:44.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2000 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 45fe3d4d-c271-4785-8d3b-d5bb998bbe9b response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68042025-04-16T04:06:45.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94e0dcaa-f9fd-4299-bcb0-3c80bd6ba11f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68052025-04-16T04:06:45.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94e0dcaa-f9fd-4299-bcb0-3c80bd6ba11f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68062025-04-16T04:06:45.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2022 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94e0dcaa-f9fd-4299-bcb0-3c80bd6ba11f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68072025-04-16T04:06:46.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2683e093-dc6a-4b19-a3d8-c774f71b9482 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68082025-04-16T04:06:46.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2683e093-dc6a-4b19-a3d8-c774f71b9482 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68092025-04-16T04:06:46.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2394 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2683e093-dc6a-4b19-a3d8-c774f71b9482 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68102025-04-16T04:06:47.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 391b840a-43d4-4bd0-bb39-657af349a458 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68112025-04-16T04:06:47.317ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 391b840a-43d4-4bd0-bb39-657af349a458 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68122025-04-16T04:06:47.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2882 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 391b840a-43d4-4bd0-bb39-657af349a458 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68132025-04-16T04:06:48.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4f2fdf6-2ac2-4bdb-a989-64e565f95274 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68142025-04-16T04:06:48.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4f2fdf6-2ac2-4bdb-a989-64e565f95274 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68152025-04-16T04:06:48.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = b4f2fdf6-2ac2-4bdb-a989-64e565f95274 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68162025-04-16T04:06:48.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 601 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 142ab8b6-a905-4d00-9bd8-5f38833aa10e response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
68172025-04-16T04:06:49.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4859d1c8-8613-4aa0-b6c9-1f4a1981914d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68182025-04-16T04:06:49.362ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4859d1c8-8613-4aa0-b6c9-1f4a1981914d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68192025-04-16T04:06:49.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4859d1c8-8613-4aa0-b6c9-1f4a1981914d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
68202025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
68212025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
68222025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68232025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68242025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
68252025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
68262025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68272025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
68282025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
68292025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
68302025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
68312025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
68322025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
68332025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68342025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68352025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
68362025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
68372025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68382025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68392025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
68402025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
68412025-04-16T04:06:49.447ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
68422025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68432025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
68442025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68452025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
68462025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68472025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
68482025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68492025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
68502025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
68512025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
68522025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
68532025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68542025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
68552025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
68562025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68572025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
68582025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68592025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
68602025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
68612025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68622025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68632025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
68642025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68652025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68662025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68672025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
68682025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
68692025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68702025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68712025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
68722025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
68732025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68742025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68752025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68762025-04-16T04:06:49.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
68772025-04-16T04:06:49.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68782025-04-16T04:06:49.450ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68792025-04-16T04:06:49.450ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.631802ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
68802025-04-16T04:06:49.450ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68812025-04-16T04:06:49.451ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
68822025-04-16T04:06:49.451ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68832025-04-16T04:06:49.451ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.04172ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
68842025-04-16T04:06:49.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68852025-04-16T04:06:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68862025-04-16T04:06:49.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68872025-04-16T04:06:49.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68882025-04-16T04:06:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
68892025-04-16T04:06:49.453ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68902025-04-16T04:06:49.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68912025-04-16T04:06:49.454ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68922025-04-16T04:06:49.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3299, tv_nsec: 736504185 } })
68932025-04-16T04:06:49.455ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3299, tv_nsec: 736512595 } })
68942025-04-16T04:06:49.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
68952025-04-16T04:06:49.456ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
68962025-04-16T04:06:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
68972025-04-16T04:06:49.456ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68982025-04-16T04:06:49.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68992025-04-16T04:06:49.457ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69002025-04-16T04:06:49.459ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69012025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69022025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69032025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69042025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
69052025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69062025-04-16T04:06:49.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69072025-04-16T04:06:49.462ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69082025-04-16T04:06:49.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69092025-04-16T04:06:49.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69102025-04-16T04:06:49.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69112025-04-16T04:06:49.465ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69122025-04-16T04:06:49.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69132025-04-16T04:06:49.467ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69142025-04-16T04:06:49.468ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69152025-04-16T04:06:49.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69162025-04-16T04:06:49.469ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69172025-04-16T04:06:49.469ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69182025-04-16T04:06:49.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69192025-04-16T04:06:49.469ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69202025-04-16T04:06:49.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69212025-04-16T04:06:49.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69222025-04-16T04:06:49.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69232025-04-16T04:06:49.495ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 48.585679ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
69242025-04-16T04:06:49.496ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 49.086038ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
69252025-04-16T04:06:49.499ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69262025-04-16T04:06:49.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69272025-04-16T04:06:49.501ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69282025-04-16T04:06:49.502ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69292025-04-16T04:06:49.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69302025-04-16T04:06:49.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69312025-04-16T04:06:49.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69322025-04-16T04:06:49.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
69332025-04-16T04:06:49.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69342025-04-16T04:06:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69352025-04-16T04:06:49.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69362025-04-16T04:06:49.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69372025-04-16T04:06:49.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69382025-04-16T04:06:49.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
69392025-04-16T04:06:49.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
69402025-04-16T04:06:49.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69412025-04-16T04:06:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
69422025-04-16T04:06:49.509ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69432025-04-16T04:06:49.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69442025-04-16T04:06:49.511ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
69452025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 63.822599ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
69462025-04-16T04:06:49.511ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69472025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
69482025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69492025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69502025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
69512025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69522025-04-16T04:06:49.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69532025-04-16T04:06:49.514ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 66.737077ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
69542025-04-16T04:06:49.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69552025-04-16T04:06:49.514ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3299, tv_nsec: 795123232 } })
69562025-04-16T04:06:49.514ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
69572025-04-16T04:06:49.515ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
69582025-04-16T04:06:49.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
69592025-04-16T04:06:49.516ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 69.175626ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
69602025-04-16T04:06:49.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
69612025-04-16T04:06:49.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69622025-04-16T04:06:49.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69632025-04-16T04:06:49.520ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 72.719495ms iteration = 26 status = 0
69642025-04-16T04:06:49.522ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
69652025-04-16T04:06:49.522ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69662025-04-16T04:06:49.522ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bdc6737-6519-4a20-a061-c6ab369debd9", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:48 GMT"} })
69672025-04-16T04:06:49.522ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69682025-04-16T04:06:49.523ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
69692025-04-16T04:06:49.523ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.561481ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
69702025-04-16T04:06:49.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69712025-04-16T04:06:49.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
69722025-04-16T04:06:49.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.52075ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
69732025-04-16T04:06:49.524ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
69742025-04-16T04:06:49.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69752025-04-16T04:06:49.526ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69762025-04-16T04:06:49.526ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 79.092641ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69772025-04-16T04:06:49.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69782025-04-16T04:06:49.535ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
69792025-04-16T04:06:49.535ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69802025-04-16T04:06:49.535ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ff1d0f-6991-4bc0-9043-6e34f9309b28", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:48 GMT"} })
69812025-04-16T04:06:49.536ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69822025-04-16T04:06:49.538ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
69832025-04-16T04:06:49.539ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69842025-04-16T04:06:49.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69852025-04-16T04:06:49.544ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69862025-04-16T04:06:49.544ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
69872025-04-16T04:06:49.544ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 97.77566ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
69882025-04-16T04:06:49.572ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69892025-04-16T04:06:49.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69902025-04-16T04:06:49.596ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.754196ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
69912025-04-16T04:06:49.610ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
69922025-04-16T04:06:49.610ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69932025-04-16T04:06:49.611ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd78df95-49ef-422e-9be2-53f2ac0d7e40", "content-length": "26", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
69942025-04-16T04:06:49.611ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69952025-04-16T04:06:49.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
69962025-04-16T04:06:49.631ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 183.741317ms iteration = 26 status = {}
69972025-04-16T04:06:49.634ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
69982025-04-16T04:06:49.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
69992025-04-16T04:06:49.637ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 190.509821ms iteration = 26 status = null
70002025-04-16T04:06:49.645ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
70012025-04-16T04:06:49.645ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70022025-04-16T04:06:49.645ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "29b4e9bf-a156-46bd-a717-7a1c5ce700e4", "content-length": "372", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70032025-04-16T04:06:49.645ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70042025-04-16T04:06:49.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70052025-04-16T04:06:49.649ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 202.672401ms iteration = 29 status = {"physical_disks_added":0}
70062025-04-16T04:06:49.689ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70072025-04-16T04:06:49.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
70082025-04-16T04:06:49.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70092025-04-16T04:06:49.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
70102025-04-16T04:06:49.693ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70112025-04-16T04:06:49.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
70122025-04-16T04:06:49.694ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70132025-04-16T04:06:49.696ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
70142025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
70152025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8317ebef-8e68-4a4f-b8c2-96fdb212eeb7", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70162025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70172025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
70182025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70192025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70202025-04-16T04:06:49.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
70212025-04-16T04:06:49.698ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70222025-04-16T04:06:49.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70232025-04-16T04:06:49.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70242025-04-16T04:06:49.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70252025-04-16T04:06:49.705ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
70262025-04-16T04:06:49.705ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70272025-04-16T04:06:49.705ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "adf6004f-6c63-4b4e-abfe-cb5955a3c014", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70282025-04-16T04:06:49.707ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 260.509792ms iteration = 27 status = {}
70292025-04-16T04:06:49.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70302025-04-16T04:06:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70312025-04-16T04:06:49.758ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
70322025-04-16T04:06:49.758ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70332025-04-16T04:06:49.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70342025-04-16T04:06:49.759ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
70352025-04-16T04:06:49.759ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3775d30-6f84-42fe-a018-220e91535ea8", "content-length": "691", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70362025-04-16T04:06:49.759ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
70372025-04-16T04:06:49.759ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
70382025-04-16T04:06:49.759ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
70392025-04-16T04:06:49.760ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 313.59678ms iteration = 27 status = {}
70402025-04-16T04:06:49.769ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
70412025-04-16T04:06:49.769ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
70422025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
70432025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
70442025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70452025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70462025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70472025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
70482025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70492025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
70502025-04-16T04:06:49.771ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
70512025-04-16T04:06:49.771ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70522025-04-16T04:06:49.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70532025-04-16T04:06:49.774ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
70542025-04-16T04:06:49.774ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
70552025-04-16T04:06:49.920ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5c7bfc28-d53e-4a7d-925e-400047dd1f35", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70562025-04-16T04:06:49.920ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
70572025-04-16T04:06:49.920ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
70582025-04-16T04:06:49.920ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfbf40e9-919c-42c1-8429-43312a1e78eb", "content-length": "682", "date": "Wed, 16 Apr 2025 04:06:49 GMT"} })
70592025-04-16T04:06:49.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70602025-04-16T04:06:49.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70612025-04-16T04:06:49.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70622025-04-16T04:06:49.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70632025-04-16T04:06:49.928ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70642025-04-16T04:06:49.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70652025-04-16T04:06:49.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70662025-04-16T04:06:49.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70672025-04-16T04:06:49.937ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 489.88401ms iteration = 26 status = {}
70682025-04-16T04:06:50.384ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffde36b8-4b35-4c37-a69c-303902ee4b66 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70692025-04-16T04:06:50.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffde36b8-4b35-4c37-a69c-303902ee4b66 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70702025-04-16T04:06:50.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2290 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ffde36b8-4b35-4c37-a69c-303902ee4b66 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70712025-04-16T04:06:51.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 453cc99a-aa79-4fc2-92b9-3fc4a61097f0 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70722025-04-16T04:06:51.406ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 453cc99a-aa79-4fc2-92b9-3fc4a61097f0 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70732025-04-16T04:06:51.408ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2726 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 453cc99a-aa79-4fc2-92b9-3fc4a61097f0 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70742025-04-16T04:06:52.429ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc800d6e-ead9-4886-9e15-cb3fc8974d9f roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70752025-04-16T04:06:52.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc800d6e-ead9-4886-9e15-cb3fc8974d9f resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70762025-04-16T04:06:52.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = bc800d6e-ead9-4886-9e15-cb3fc8974d9f response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70772025-04-16T04:06:53.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a21f5ca-48db-4a79-98c4-48598e155836 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70782025-04-16T04:06:53.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a21f5ca-48db-4a79-98c4-48598e155836 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70792025-04-16T04:06:53.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2496 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 1a21f5ca-48db-4a79-98c4-48598e155836 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70802025-04-16T04:06:54.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4078127-c3e7-4845-ab35-27009547824d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70812025-04-16T04:06:54.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4078127-c3e7-4845-ab35-27009547824d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70822025-04-16T04:06:54.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f4078127-c3e7-4845-ab35-27009547824d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70832025-04-16T04:06:55.497ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b3c7f4-f28c-475f-81a5-e7a03f199586 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70842025-04-16T04:06:55.498ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b3c7f4-f28c-475f-81a5-e7a03f199586 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70852025-04-16T04:06:55.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 69b3c7f4-f28c-475f-81a5-e7a03f199586 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70862025-04-16T04:06:56.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 905ce237-1204-4633-90be-6d671cdbb5c6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70872025-04-16T04:06:56.520ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 905ce237-1204-4633-90be-6d671cdbb5c6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70882025-04-16T04:06:56.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 905ce237-1204-4633-90be-6d671cdbb5c6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70892025-04-16T04:06:57.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cb9c47b-1a3e-4bd1-9869-be25b634f3d7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70902025-04-16T04:06:57.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cb9c47b-1a3e-4bd1-9869-be25b634f3d7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70912025-04-16T04:06:57.544ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2597 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5cb9c47b-1a3e-4bd1-9869-be25b634f3d7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70922025-04-16T04:06:58.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 212d30c7-c80f-4f66-8348-ae9fa02eddee roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70932025-04-16T04:06:58.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 212d30c7-c80f-4f66-8348-ae9fa02eddee resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70942025-04-16T04:06:58.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1582 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 212d30c7-c80f-4f66-8348-ae9fa02eddee response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
70952025-04-16T04:06:58.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 517 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 809b127b-322f-4d3a-9c15-6fb4a6ea38e5 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
70962025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
70972025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70982025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
70992025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71002025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71012025-04-16T04:06:59.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71022025-04-16T04:06:59.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71032025-04-16T04:06:59.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71042025-04-16T04:06:59.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71052025-04-16T04:06:59.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71062025-04-16T04:06:59.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71072025-04-16T04:06:59.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71082025-04-16T04:06:59.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71092025-04-16T04:06:59.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
71102025-04-16T04:06:59.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71112025-04-16T04:06:59.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71122025-04-16T04:06:59.469ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
71132025-04-16T04:06:59.469ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.014325ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
71142025-04-16T04:06:59.478ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71152025-04-16T04:06:59.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71162025-04-16T04:06:59.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.182698ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71172025-04-16T04:06:59.584ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f52f452f-e4aa-4b42-a57d-7a5a0c95a0d2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71182025-04-16T04:06:59.584ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f52f452f-e4aa-4b42-a57d-7a5a0c95a0d2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71192025-04-16T04:06:59.586ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2125 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f52f452f-e4aa-4b42-a57d-7a5a0c95a0d2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71202025-04-16T04:07:00.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 57ac47a1-576c-4879-a621-9aa8a3ad1938 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71212025-04-16T04:07:00.606ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 57ac47a1-576c-4879-a621-9aa8a3ad1938 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71222025-04-16T04:07:00.608ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2200 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 57ac47a1-576c-4879-a621-9aa8a3ad1938 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71232025-04-16T04:07:01.629ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a585952-31a0-4424-9e93-d96a0e92e194 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71242025-04-16T04:07:01.629ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a585952-31a0-4424-9e93-d96a0e92e194 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71252025-04-16T04:07:01.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2812 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a585952-31a0-4424-9e93-d96a0e92e194 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71262025-04-16T04:07:02.651ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10a57b53-7766-49f3-9e51-c9a9ebef587a roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71272025-04-16T04:07:02.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10a57b53-7766-49f3-9e51-c9a9ebef587a resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71282025-04-16T04:07:02.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 10a57b53-7766-49f3-9e51-c9a9ebef587a response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71292025-04-16T04:07:03.672ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4fdc5835-e817-4bc1-8c8e-79c8dcd357d7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71302025-04-16T04:07:03.673ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4fdc5835-e817-4bc1-8c8e-79c8dcd357d7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71312025-04-16T04:07:03.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2455 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4fdc5835-e817-4bc1-8c8e-79c8dcd357d7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71322025-04-16T04:07:04.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd285f13-c0fa-4ff4-8e3e-5dadc5d6d041 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71332025-04-16T04:07:04.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd285f13-c0fa-4ff4-8e3e-5dadc5d6d041 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71342025-04-16T04:07:04.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2325 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cd285f13-c0fa-4ff4-8e3e-5dadc5d6d041 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71352025-04-16T04:07:05.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e407b482-6eed-4b74-b8b6-3a3e7049be04 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71362025-04-16T04:07:05.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e407b482-6eed-4b74-b8b6-3a3e7049be04 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71372025-04-16T04:07:05.718ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = e407b482-6eed-4b74-b8b6-3a3e7049be04 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71382025-04-16T04:07:06.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d59138a6-ebca-4be3-8ae6-86b295275646 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71392025-04-16T04:07:06.737ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d59138a6-ebca-4be3-8ae6-86b295275646 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71402025-04-16T04:07:06.739ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d59138a6-ebca-4be3-8ae6-86b295275646 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71412025-04-16T04:07:07.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5c71e63-30c5-4c5d-bf97-951cac70c520 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71422025-04-16T04:07:07.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5c71e63-30c5-4c5d-bf97-951cac70c520 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71432025-04-16T04:07:07.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2430 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d5c71e63-30c5-4c5d-bf97-951cac70c520 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71442025-04-16T04:07:08.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 592 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 3c9acb96-8273-47d8-857c-ccced5c34b20 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
71452025-04-16T04:07:08.781ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a99e85a-a703-4f66-96ad-845550339474 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71462025-04-16T04:07:08.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a99e85a-a703-4f66-96ad-845550339474 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71472025-04-16T04:07:08.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2461 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a99e85a-a703-4f66-96ad-845550339474 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71482025-04-16T04:07:09.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
71492025-04-16T04:07:09.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
71502025-04-16T04:07:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71512025-04-16T04:07:09.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71522025-04-16T04:07:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71532025-04-16T04:07:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71542025-04-16T04:07:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71552025-04-16T04:07:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71562025-04-16T04:07:09.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71572025-04-16T04:07:09.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71582025-04-16T04:07:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71592025-04-16T04:07:09.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71602025-04-16T04:07:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71612025-04-16T04:07:09.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
71622025-04-16T04:07:09.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71632025-04-16T04:07:09.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71642025-04-16T04:07:09.466ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
71652025-04-16T04:07:09.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.018115ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
71662025-04-16T04:07:09.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71672025-04-16T04:07:09.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71682025-04-16T04:07:09.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.556063ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71692025-04-16T04:07:09.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94a35368-633f-412c-b930-803ccc92bf99 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71702025-04-16T04:07:09.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94a35368-633f-412c-b930-803ccc92bf99 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71712025-04-16T04:07:09.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1745 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 94a35368-633f-412c-b930-803ccc92bf99 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71722025-04-16T04:07:10.825ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb31ab38-06ad-472f-9119-a64a201df006 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71732025-04-16T04:07:10.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb31ab38-06ad-472f-9119-a64a201df006 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71742025-04-16T04:07:10.828ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2364 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = fb31ab38-06ad-472f-9119-a64a201df006 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71752025-04-16T04:07:11.847ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0218a40d-9bf1-48c7-b84e-265abb7cf527 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71762025-04-16T04:07:11.848ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0218a40d-9bf1-48c7-b84e-265abb7cf527 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71772025-04-16T04:07:11.849ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2092 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0218a40d-9bf1-48c7-b84e-265abb7cf527 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71782025-04-16T04:07:12.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803c1948-3953-41c7-8960-1be79253c9a6 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71792025-04-16T04:07:12.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803c1948-3953-41c7-8960-1be79253c9a6 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71802025-04-16T04:07:12.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2310 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 803c1948-3953-41c7-8960-1be79253c9a6 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71812025-04-16T04:07:13.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 902d517d-dfdf-4198-b5ea-d419b5bb6e10 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
71822025-04-16T04:07:13.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 902d517d-dfdf-4198-b5ea-d419b5bb6e10 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
71832025-04-16T04:07:13.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2585 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 902d517d-dfdf-4198-b5ea-d419b5bb6e10 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
71842025-04-16T04:07:13.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 6af10c0d-3ca7-4895-92b2-ef54b50c3af6 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
71852025-04-16T04:07:13.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 6af10c0d-3ca7-4895-92b2-ef54b50c3af6 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
71862025-04-16T04:07:13.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2463 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = 6af10c0d-3ca7-4895-92b2-ef54b50c3af6 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
71872025-04-16T04:07:13.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e87bd23-30f2-48b5-95ff-14b3ee284bd4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71882025-04-16T04:07:13.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e87bd23-30f2-48b5-95ff-14b3ee284bd4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71892025-04-16T04:07:13.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2821 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 2e87bd23-30f2-48b5-95ff-14b3ee284bd4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71902025-04-16T04:07:14.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ed26720-8112-43ff-b8ab-5e0bf7cde8ea roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71912025-04-16T04:07:14.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ed26720-8112-43ff-b8ab-5e0bf7cde8ea resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71922025-04-16T04:07:14.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2123 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 8ed26720-8112-43ff-b8ab-5e0bf7cde8ea response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71932025-04-16T04:07:15.937ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d2863e7-c9f9-42c0-a2b4-33f58fad79e2 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71942025-04-16T04:07:15.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d2863e7-c9f9-42c0-a2b4-33f58fad79e2 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71952025-04-16T04:07:15.939ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2401 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3d2863e7-c9f9-42c0-a2b4-33f58fad79e2 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71962025-04-16T04:07:16.959ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a4731b4-fa77-42da-9eba-f90925036ace roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71972025-04-16T04:07:16.959ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a4731b4-fa77-42da-9eba-f90925036ace resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71982025-04-16T04:07:16.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2517 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 9a4731b4-fa77-42da-9eba-f90925036ace response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
71992025-04-16T04:07:17.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d977c0ca-fdd8-4d8c-afbb-c06e4dc916c5 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72002025-04-16T04:07:17.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d977c0ca-fdd8-4d8c-afbb-c06e4dc916c5 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72012025-04-16T04:07:17.983ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2538 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = d977c0ca-fdd8-4d8c-afbb-c06e4dc916c5 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72022025-04-16T04:07:18.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 509 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 1e7861b7-26c7-47e2-a16c-9f0b7df838b4 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
72032025-04-16T04:07:19.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a4624e7-5875-4538-829e-f695f0eb09b3 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72042025-04-16T04:07:19.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a4624e7-5875-4538-829e-f695f0eb09b3 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72052025-04-16T04:07:19.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2646 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 3a4624e7-5875-4538-829e-f695f0eb09b3 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
72062025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
72072025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
72082025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72092025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
72102025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
72112025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
72122025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
72132025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
72142025-04-16T04:07:19.446ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
72152025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72162025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
72172025-04-16T04:07:19.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72182025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
72192025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72202025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
72212025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
72222025-04-16T04:07:19.447ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
72232025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
72242025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72252025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
72262025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
72272025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72282025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
72292025-04-16T04:07:19.447ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c
72302025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
72312025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.231\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c"}
72322025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72332025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
72342025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
72352025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72362025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
72372025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
72382025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72392025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
72402025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72412025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
72422025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
72432025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72442025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72452025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72462025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
72472025-04-16T04:07:19.447ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-16 03:57:19.447500903 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
72482025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
72492025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
72502025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
72512025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
72522025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
72532025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
72542025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72552025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72562025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
72572025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
72582025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
72592025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72602025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
72612025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
72622025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
72632025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
72642025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
72652025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
72662025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
72672025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
72682025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
72692025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72702025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
72712025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
72722025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
72732025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
72742025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
72752025-04-16T04:07:19.447ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
72762025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72772025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
72782025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
72792025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72802025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
72812025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
72822025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
72832025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72842025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72852025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72862025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72872025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
72882025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
72892025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
72902025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72912025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72922025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
72932025-04-16T04:07:19.448ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("451c72b9-d66e-4317-9544-47f4e8cab312.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5776912e-8bfe-41f6-b8b6-5c723f305e7d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a548bab8-a837-4a11-a922-c9029e5c0fbe.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 3329, tv_nsec: 728930209 } })
72942025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
72952025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
72962025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
72972025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72982025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
72992025-04-16T04:07:19.448ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("97388ba5-45a6-4ce2-ac1c-18f5fc13fe23.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ead8dc35-753d-459f-bec4-f8ca9d19736f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3329, tv_nsec: 729013791 } })
73002025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
73012025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.129892ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
73022025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
73032025-04-16T04:07:19.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.00257ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
73042025-04-16T04:07:19.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73052025-04-16T04:07:19.450ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73062025-04-16T04:07:19.450ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73072025-04-16T04:07:19.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
73082025-04-16T04:07:19.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73092025-04-16T04:07:19.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73102025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
73112025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73122025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73132025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
73142025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73152025-04-16T04:07:19.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73162025-04-16T04:07:19.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73172025-04-16T04:07:19.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73182025-04-16T04:07:19.457ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-16 03:57:19.447500903 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
73192025-04-16T04:07:19.457ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.60704ms iteration = 14 status = {"expiration":"2025-04-16T03:57:19.447500903Z","pruned":[]}
73202025-04-16T04:07:19.457ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.874865ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
73212025-04-16T04:07:19.457ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73222025-04-16T04:07:19.459ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3329, tv_nsec: 740157671 } })
73232025-04-16T04:07:19.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
73242025-04-16T04:07:19.459ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
73252025-04-16T04:07:19.459ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1086
73262025-04-16T04:07:19.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
73272025-04-16T04:07:19.460ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
73282025-04-16T04:07:19.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
73292025-04-16T04:07:19.462ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.538137ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
73302025-04-16T04:07:19.463ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73312025-04-16T04:07:19.464ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3329, tv_nsec: 745497636 } })
73322025-04-16T04:07:19.464ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73332025-04-16T04:07:19.465ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1086
73342025-04-16T04:07:19.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73352025-04-16T04:07:19.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
73362025-04-16T04:07:19.467ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
73372025-04-16T04:07:19.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
73382025-04-16T04:07:19.472ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73392025-04-16T04:07:19.473ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73402025-04-16T04:07:19.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73412025-04-16T04:07:19.474ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
73422025-04-16T04:07:19.474ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73432025-04-16T04:07:19.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73442025-04-16T04:07:19.479ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-fe2a235d-b62d-4168-a36a-687174761bf3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3329, tv_nsec: 759882700 } })
73452025-04-16T04:07:19.479ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73462025-04-16T04:07:19.479ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1086 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
73472025-04-16T04:07:19.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73482025-04-16T04:07:19.480ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
73492025-04-16T04:07:19.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
73502025-04-16T04:07:19.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73512025-04-16T04:07:19.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
73522025-04-16T04:07:19.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73532025-04-16T04:07:19.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
73542025-04-16T04:07:19.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73552025-04-16T04:07:19.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73562025-04-16T04:07:19.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73572025-04-16T04:07:19.490ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73582025-04-16T04:07:19.491ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
73592025-04-16T04:07:19.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
73602025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
73612025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
73622025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73632025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73642025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
73652025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73662025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
73672025-04-16T04:07:19.494ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
73682025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 48.011758ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
73692025-04-16T04:07:19.494ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73702025-04-16T04:07:19.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73712025-04-16T04:07:19.495ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73722025-04-16T04:07:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
73732025-04-16T04:07:19.497ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
73742025-04-16T04:07:19.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
73752025-04-16T04:07:19.497ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73762025-04-16T04:07:19.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73772025-04-16T04:07:19.499ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73782025-04-16T04:07:19.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73792025-04-16T04:07:19.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73802025-04-16T04:07:19.500ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
73812025-04-16T04:07:19.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
73822025-04-16T04:07:19.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
73832025-04-16T04:07:19.501ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73842025-04-16T04:07:19.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73852025-04-16T04:07:19.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73862025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
73872025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73882025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73892025-04-16T04:07:19.506ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73902025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
73912025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73922025-04-16T04:07:19.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
73932025-04-16T04:07:19.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.327701ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
73942025-04-16T04:07:19.508ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 61.111776ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
73952025-04-16T04:07:19.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73962025-04-16T04:07:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
73972025-04-16T04:07:19.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73982025-04-16T04:07:19.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73992025-04-16T04:07:19.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74002025-04-16T04:07:19.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74012025-04-16T04:07:19.519ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
74022025-04-16T04:07:19.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 72.4175ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
74032025-04-16T04:07:19.521ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.894229ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
74042025-04-16T04:07:19.523ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
74052025-04-16T04:07:19.523ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 75.867178ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
74062025-04-16T04:07:19.534ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
74072025-04-16T04:07:19.534ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
74082025-04-16T04:07:19.534ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
74092025-04-16T04:07:19.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
74102025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
74112025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
74122025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
74132025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
74142025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74152025-04-16T04:07:19.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74162025-04-16T04:07:19.539ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74172025-04-16T04:07:19.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74182025-04-16T04:07:19.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.434163ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
74192025-04-16T04:07:19.542ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
74202025-04-16T04:07:19.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
74212025-04-16T04:07:19.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
74222025-04-16T04:07:19.543ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
74232025-04-16T04:07:19.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
74242025-04-16T04:07:19.544ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
74252025-04-16T04:07:19.544ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
74262025-04-16T04:07:19.544ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 97.026516ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
74272025-04-16T04:07:19.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
74282025-04-16T04:07:19.544ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74292025-04-16T04:07:19.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74302025-04-16T04:07:19.545ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74312025-04-16T04:07:19.547ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 100.301049ms iteration = 30 status = {"physical_disks_added":0}
74322025-04-16T04:07:19.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
74332025-04-16T04:07:19.549ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
74342025-04-16T04:07:19.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
74352025-04-16T04:07:19.550ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
74362025-04-16T04:07:19.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
74372025-04-16T04:07:19.551ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 104.280858ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
74382025-04-16T04:07:19.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74392025-04-16T04:07:19.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74402025-04-16T04:07:19.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
74412025-04-16T04:07:19.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.721066ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74422025-04-16T04:07:19.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
74432025-04-16T04:07:19.556ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74442025-04-16T04:07:19.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74452025-04-16T04:07:19.557ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74462025-04-16T04:07:19.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
74472025-04-16T04:07:19.560ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
74482025-04-16T04:07:19.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
74492025-04-16T04:07:19.561ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74502025-04-16T04:07:19.561ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
74512025-04-16T04:07:19.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
74522025-04-16T04:07:19.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
74532025-04-16T04:07:19.562ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74542025-04-16T04:07:19.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74552025-04-16T04:07:19.569ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-16T03:54:13.341782Z, time_modified: 2025-04-16T03:54:13.341782Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, db_silo: Silo { identity: SiloIdentity { id: 34f3b91a-eb8a-4b88-8074-a57ae3591f56, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-04-16T03:54:21.972363Z, time_modified: 2025-04-16T03:54:21.972363Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00" }] }) }
74562025-04-16T04:07:19.569ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 122.115191ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]}},"default_endpoint":{"silo_id":"34f3b91a-eb8a-4b88-8074-a57ae3591f56","tls_certs":["b37cd9c1f4d1e22ae0f01c4bb81b0727c7445a9a8c97853520f9f4c9a44e8c00"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
74572025-04-16T04:07:19.581ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
74582025-04-16T04:07:19.610ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
74592025-04-16T04:07:19.610ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
74602025-04-16T04:07:19.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74612025-04-16T04:07:19.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74622025-04-16T04:07:19.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74632025-04-16T04:07:19.613ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
74642025-04-16T04:07:19.613ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 0 version = 2
74652025-04-16T04:07:19.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
74662025-04-16T04:07:19.613ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
74672025-04-16T04:07:19.613ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 165.972226ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
74682025-04-16T04:07:19.614ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74692025-04-16T04:07:19.614ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
74702025-04-16T04:07:19.614ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 166.357654ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
74712025-04-16T04:07:19.614ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 166.287142ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
74722025-04-16T04:07:19.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
74732025-04-16T04:07:19.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
74742025-04-16T04:07:19.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74752025-04-16T04:07:19.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74762025-04-16T04:07:19.618ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
74772025-04-16T04:07:19.619ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74782025-04-16T04:07:19.619ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9af1db1c-79e8-40a6-b29f-587375c605e2", "content-length": "26", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
74792025-04-16T04:07:19.619ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74802025-04-16T04:07:19.621ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 994677de-3dfa-42cf-a42e-f76cebd318ef dns_zone_name = control-plane.oxide.internal found_so_far = 54 version = 1
74812025-04-16T04:07:19.621ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
74822025-04-16T04:07:19.621ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
74832025-04-16T04:07:19.621ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-04-16 03:54:19.362289 UTC dns_group = internal generation = 1
74842025-04-16T04:07:19.621ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 174.026055ms iteration = 14 status = {"generation":1}
74852025-04-16T04:07:19.626ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74862025-04-16T04:07:19.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: fe2a235d-b62d-4168-a36a-687174761bf3, lookup_type: ById(fe2a235d-b62d-4168-a36a-687174761bf3) } result = Ok(())
74872025-04-16T04:07:19.631ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74882025-04-16T04:07:19.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
74892025-04-16T04:07:19.659ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1094 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 zone_address = fd00:1122:3344:101::2
74902025-04-16T04:07:19.659ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74912025-04-16T04:07:19.660ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c684dd18-b59a-42da-9441-2105acb4343b", "content-length": "26", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
74922025-04-16T04:07:19.660ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1097 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74932025-04-16T04:07:19.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74942025-04-16T04:07:19.662ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
74952025-04-16T04:07:19.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
74962025-04-16T04:07:19.673ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
74972025-04-16T04:07:19.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74982025-04-16T04:07:19.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
74992025-04-16T04:07:19.688ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
75002025-04-16T04:07:19.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
75012025-04-16T04:07:19.701ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
75022025-04-16T04:07:19.701ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 254.138156ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
75032025-04-16T04:07:19.747ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
75042025-04-16T04:07:19.747ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
75052025-04-16T04:07:19.747ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2faf2235-db3e-4df9-9e28-f8720ff73e89", "content-length": "372", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75062025-04-16T04:07:19.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75072025-04-16T04:07:19.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75082025-04-16T04:07:19.749ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 301.97176ms iteration = 27 status = 0
75092025-04-16T04:07:19.751ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a89ca018-fc9d-4e74-9816-9f598f318c3f dns_zone_name = oxide.test found_so_far = 1 version = 2
75102025-04-16T04:07:19.751ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
75112025-04-16T04:07:19.751ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-04-16 03:54:22.047720 UTC dns_group = external generation = 2
75122025-04-16T04:07:19.751ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 303.734585ms iteration = 14 status = {"generation":2}
75132025-04-16T04:07:19.754ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
75142025-04-16T04:07:19.763ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 315.859154ms iteration = 27 status = {}
75152025-04-16T04:07:19.788ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75162025-04-16T04:07:19.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75172025-04-16T04:07:19.790ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75182025-04-16T04:07:19.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75192025-04-16T04:07:19.791ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
75202025-04-16T04:07:19.791ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
75212025-04-16T04:07:19.792ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6468a165-9cae-450a-9967-603035b4894e", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75222025-04-16T04:07:19.792ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75232025-04-16T04:07:19.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75242025-04-16T04:07:19.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75252025-04-16T04:07:19.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75262025-04-16T04:07:19.794ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75272025-04-16T04:07:19.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75282025-04-16T04:07:19.795ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75292025-04-16T04:07:19.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75302025-04-16T04:07:19.799ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75312025-04-16T04:07:19.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75322025-04-16T04:07:19.803ZINFOnexus (BackgroundTasks): VPC route manager sled fe2a235d-b62d-4168-a36a-687174761bf3 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
75332025-04-16T04:07:19.803ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
75342025-04-16T04:07:19.803ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e6f1ad49-b15e-42e7-8d73-c740f697fbef", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75352025-04-16T04:07:19.805ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 358.491156ms iteration = 28 status = {}
75362025-04-16T04:07:19.813ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1094 zone_address = fd00:1122:3344:101::2
75372025-04-16T04:07:19.813ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
75382025-04-16T04:07:19.814ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2f39246-4bc5-429f-990a-4d862d0e7970", "content-length": "26", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75392025-04-16T04:07:19.814ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1097 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
75402025-04-16T04:07:19.816ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
75412025-04-16T04:07:19.875ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
75422025-04-16T04:07:19.876ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "dbcf2191-755c-4467-8a9b-4917045a20f8", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75432025-04-16T04:07:19.876ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75442025-04-16T04:07:19.878ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d757ffed-d7dd-4ccb-867b-49f4453fa845", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75452025-04-16T04:07:19.878ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75462025-04-16T04:07:19.881ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "58afca21-394c-4bd0-bf44-37759c4d47fd", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75472025-04-16T04:07:19.881ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75482025-04-16T04:07:19.883ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
75492025-04-16T04:07:19.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 437.030336ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
75502025-04-16T04:07:19.887ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
75512025-04-16T04:07:19.887ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "6bf27530-a013-471c-90b4-c63b4d5273aa", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75522025-04-16T04:07:19.887ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75532025-04-16T04:07:19.890ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "51357d3a-3f9e-4ac2-be7e-f8f0010ca12b", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75542025-04-16T04:07:19.890ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75552025-04-16T04:07:19.892ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75562025-04-16T04:07:19.892ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 445.272549ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
75572025-04-16T04:07:19.944ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
75582025-04-16T04:07:19.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
75592025-04-16T04:07:19.956ZDEBGnexus (BackgroundTasks): client request SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
75602025-04-16T04:07:19.957ZDEBGnexus (BackgroundTasks): client response SledAgent = fe2a235d-b62d-4168-a36a-687174761bf3 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "58b594a3-f6c3-4de1-af1f-d9ed3233c482", "content-length": "691", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75612025-04-16T04:07:19.957ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
75622025-04-16T04:07:19.957ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
75632025-04-16T04:07:19.957ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
75642025-04-16T04:07:19.959ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 512.197319ms iteration = 28 status = {}
75652025-04-16T04:07:19.959ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 512.17277ms iteration = 27 status = null
75662025-04-16T04:07:19.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75672025-04-16T04:07:19.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75682025-04-16T04:07:19.976ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 9af8862d-00a7-42cc-a02f-00a5cd56f8f1, rack_id: e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(7bd123e3-15a4-4aad-a21d-91dd9989b47d) } rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
75692025-04-16T04:07:19.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75702025-04-16T04:07:19.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75712025-04-16T04:07:19.987ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1833 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
75722025-04-16T04:07:19.987ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1860 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 7bd123e3-15a4-4aad-a21d-91dd9989b47d
75732025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
75742025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
75752025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75762025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75772025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75782025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2007 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
75792025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
75802025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2043 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
75812025-04-16T04:07:19.988ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1
75822025-04-16T04:07:19.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75832025-04-16T04:07:19.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75842025-04-16T04:07:19.991ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch_location = Switch0
75852025-04-16T04:07:19.991ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
75862025-04-16T04:07:20.094ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "67765a08-eece-428b-85fc-9bd10015e39f", "date": "Wed, 16 Apr 2025 04:07:19 GMT"} })
75872025-04-16T04:07:20.094ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 switch = Switch0
75882025-04-16T04:07:20.094ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
75892025-04-16T04:07:20.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = de0f0ad5-7f04-4db3-b8c0-97f3839bbe00 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
75902025-04-16T04:07:20.095ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e3d6bb73-9f1f-4c39-91ee-52bec0a0bab1 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8188a6d2-6444-4ca3-8c61-d1a469c2cc07", "content-length": "682", "date": "Wed, 16 Apr 2025 04:07:20 GMT"} })
75912025-04-16T04:07:20.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75922025-04-16T04:07:20.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = de0f0ad5-7f04-4db3-b8c0-97f3839bbe00 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
75932025-04-16T04:07:20.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75942025-04-16T04:07:20.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = de0f0ad5-7f04-4db3-b8c0-97f3839bbe00 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
75952025-04-16T04:07:20.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75962025-04-16T04:07:20.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75972025-04-16T04:07:20.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75982025-04-16T04:07:20.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75992025-04-16T04:07:20.106ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76002025-04-16T04:07:20.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76012025-04-16T04:07:20.113ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 666.856923ms iteration = 27 status = {}
76022025-04-16T04:07:21.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 632575ba-5c40-4bd6-88f2-28821f2874a1 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76032025-04-16T04:07:21.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 632575ba-5c40-4bd6-88f2-28821f2874a1 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76042025-04-16T04:07:21.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 632575ba-5c40-4bd6-88f2-28821f2874a1 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76052025-04-16T04:07:22.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a623811-f0bb-4f2a-831a-5006479bf676 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76062025-04-16T04:07:22.142ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a623811-f0bb-4f2a-831a-5006479bf676 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76072025-04-16T04:07:22.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2726 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6a623811-f0bb-4f2a-831a-5006479bf676 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76082025-04-16T04:07:23.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4234a32a-1bd2-4d92-84e9-4f89a9c85aac roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76092025-04-16T04:07:23.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4234a32a-1bd2-4d92-84e9-4f89a9c85aac resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76102025-04-16T04:07:23.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3068 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 4234a32a-1bd2-4d92-84e9-4f89a9c85aac response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76112025-04-16T04:07:24.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18eb4ab4-0630-4a00-8535-710c1d0c1510 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76122025-04-16T04:07:24.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18eb4ab4-0630-4a00-8535-710c1d0c1510 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76132025-04-16T04:07:24.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 18eb4ab4-0630-4a00-8535-710c1d0c1510 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76142025-04-16T04:07:25.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6dff6f5b-10b9-4af9-9771-b8f2fa72cc4c roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76152025-04-16T04:07:25.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6dff6f5b-10b9-4af9-9771-b8f2fa72cc4c resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76162025-04-16T04:07:25.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6dff6f5b-10b9-4af9-9771-b8f2fa72cc4c response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76172025-04-16T04:07:26.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f61fa971-a31c-496e-9f76-ddbd7946587d roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76182025-04-16T04:07:26.233ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f61fa971-a31c-496e-9f76-ddbd7946587d resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76192025-04-16T04:07:26.235ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f61fa971-a31c-496e-9f76-ddbd7946587d response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76202025-04-16T04:07:27.255ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbcbc0d1-65a9-404c-b7e3-83c99f7ddf78 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76212025-04-16T04:07:27.255ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbcbc0d1-65a9-404c-b7e3-83c99f7ddf78 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76222025-04-16T04:07:27.257ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2431 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = dbcbc0d1-65a9-404c-b7e3-83c99f7ddf78 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76232025-04-16T04:07:28.276ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c1db61fd-021a-4f44-bb8a-4bbf4c6982b4 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76242025-04-16T04:07:28.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c1db61fd-021a-4f44-bb8a-4bbf4c6982b4 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76252025-04-16T04:07:28.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2253 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = c1db61fd-021a-4f44-bb8a-4bbf4c6982b4 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76262025-04-16T04:07:28.566ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 485 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = ee6606d1-045e-4e10-a33d-5b3a1498c00d response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
76272025-04-16T04:07:29.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f1bd515f-965b-410f-a671-34c93626e423 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76282025-04-16T04:07:29.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f1bd515f-965b-410f-a671-34c93626e423 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76292025-04-16T04:07:29.300ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2453 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f1bd515f-965b-410f-a671-34c93626e423 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76302025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
76312025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76322025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
76332025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76342025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76352025-04-16T04:07:29.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76362025-04-16T04:07:29.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76372025-04-16T04:07:29.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76382025-04-16T04:07:29.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76392025-04-16T04:07:29.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76402025-04-16T04:07:29.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76412025-04-16T04:07:29.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76422025-04-16T04:07:29.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76432025-04-16T04:07:29.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76442025-04-16T04:07:29.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76452025-04-16T04:07:29.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
76462025-04-16T04:07:29.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76472025-04-16T04:07:29.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76482025-04-16T04:07:29.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
76492025-04-16T04:07:29.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.021037ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
76502025-04-16T04:07:29.516ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.997691ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76512025-04-16T04:07:30.321ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0d8c67c0-efe5-47c3-b874-ed9dcc3a506e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76522025-04-16T04:07:30.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0d8c67c0-efe5-47c3-b874-ed9dcc3a506e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76532025-04-16T04:07:30.323ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1724 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 0d8c67c0-efe5-47c3-b874-ed9dcc3a506e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76542025-04-16T04:07:31.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1f899d-91bc-42cb-8256-37be4764f5bb roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76552025-04-16T04:07:31.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1f899d-91bc-42cb-8256-37be4764f5bb resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76562025-04-16T04:07:31.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2674 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ce1f899d-91bc-42cb-8256-37be4764f5bb response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76572025-04-16T04:07:32.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 738df812-cafd-4a5d-9c34-11373bf89a13 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76582025-04-16T04:07:32.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 738df812-cafd-4a5d-9c34-11373bf89a13 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76592025-04-16T04:07:32.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2236 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 738df812-cafd-4a5d-9c34-11373bf89a13 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76602025-04-16T04:07:33.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ae30dec7-cd2d-4721-9474-7b52d9fd9454 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76612025-04-16T04:07:33.389ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ae30dec7-cd2d-4721-9474-7b52d9fd9454 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76622025-04-16T04:07:33.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2064 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ae30dec7-cd2d-4721-9474-7b52d9fd9454 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76632025-04-16T04:07:34.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5525f33-3ce6-46f9-b4b0-ca731cefd975 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76642025-04-16T04:07:34.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5525f33-3ce6-46f9-b4b0-ca731cefd975 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76652025-04-16T04:07:34.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2406 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = f5525f33-3ce6-46f9-b4b0-ca731cefd975 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76662025-04-16T04:07:35.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 00b4a757-070f-4cbb-b2a8-58d65b33e089 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76672025-04-16T04:07:35.431ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 00b4a757-070f-4cbb-b2a8-58d65b33e089 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76682025-04-16T04:07:35.433ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2068 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 00b4a757-070f-4cbb-b2a8-58d65b33e089 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76692025-04-16T04:07:36.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f14abba-9e15-44e6-bfcd-bcf700267164 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76702025-04-16T04:07:36.454ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f14abba-9e15-44e6-bfcd-bcf700267164 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76712025-04-16T04:07:36.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 5f14abba-9e15-44e6-bfcd-bcf700267164 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76722025-04-16T04:07:37.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a8b554d-9bd6-414f-a5dd-e169dda6a265 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76732025-04-16T04:07:37.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a8b554d-9bd6-414f-a5dd-e169dda6a265 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76742025-04-16T04:07:37.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2225 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 7a8b554d-9bd6-414f-a5dd-e169dda6a265 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76752025-04-16T04:07:38.498ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afab7979-38d2-4218-9bd2-b21854a0e0fc roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76762025-04-16T04:07:38.498ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afab7979-38d2-4218-9bd2-b21854a0e0fc resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76772025-04-16T04:07:38.500ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = afab7979-38d2-4218-9bd2-b21854a0e0fc response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
76782025-04-16T04:07:38.567ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 568 local_addr = [fd00:1122:3344:101::b]:55627 method = GET remote_addr = [fd00:1122:3344:101::d]:63121 req_id = 8db84d45-fe95-4802-bcbe-38170abb3368 response_code = 200 uri = /9fa72b7d-ab32-4040-b078-1b91700ee410
76792025-04-16T04:07:39.446ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
76802025-04-16T04:07:39.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76812025-04-16T04:07:39.446ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
76822025-04-16T04:07:39.446ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76832025-04-16T04:07:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76842025-04-16T04:07:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76852025-04-16T04:07:39.477ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76862025-04-16T04:07:39.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76872025-04-16T04:07:39.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76882025-04-16T04:07:39.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.389879ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76892025-04-16T04:07:39.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76902025-04-16T04:07:39.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76912025-04-16T04:07:39.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76922025-04-16T04:07:39.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76932025-04-16T04:07:39.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76942025-04-16T04:07:39.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76952025-04-16T04:07:39.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 52b2be05-d033-410b-ab4c-8c33155cdd4c, lookup_type: ById(52b2be05-d033-410b-ab4c-8c33155cdd4c) } result = Ok(())
76962025-04-16T04:07:39.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76972025-04-16T04:07:39.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76982025-04-16T04:07:39.493ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c original_time_created = 2025-04-16 03:54:06.544998 UTC target_id = 52b2be05-d033-410b-ab4c-8c33155cdd4c time_created = 2025-04-16 03:54:06.544998 UTC
76992025-04-16T04:07:39.493ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.61055ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"52b2be05-d033-410b-ab4c-8c33155cdd4c","time_created":"2025-04-16T03:54:06.544998Z"}
77002025-04-16T04:07:39.520ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44467cf8-fb70-4494-8dca-7f10e3118d20 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77012025-04-16T04:07:39.520ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44467cf8-fb70-4494-8dca-7f10e3118d20 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77022025-04-16T04:07:39.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1422 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 44467cf8-fb70-4494-8dca-7f10e3118d20 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77032025-04-16T04:07:40.541ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6abb1304-62cb-4cff-a4a1-6ae66f6ac5f7 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77042025-04-16T04:07:40.541ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6abb1304-62cb-4cff-a4a1-6ae66f6ac5f7 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77052025-04-16T04:07:40.543ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2344 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 6abb1304-62cb-4cff-a4a1-6ae66f6ac5f7 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77062025-04-16T04:07:41.562ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac4732ac-64eb-41ab-aaab-cb93b4ec2b90 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77072025-04-16T04:07:41.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac4732ac-64eb-41ab-aaab-cb93b4ec2b90 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77082025-04-16T04:07:41.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2801 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = ac4732ac-64eb-41ab-aaab-cb93b4ec2b90 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77092025-04-16T04:07:42.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc271633-34ec-42e5-a96a-600d86471ae8 roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77102025-04-16T04:07:42.585ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc271633-34ec-42e5-a96a-600d86471ae8 resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77112025-04-16T04:07:42.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = cc271633-34ec-42e5-a96a-600d86471ae8 response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77122025-04-16T04:07:43.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = ade61282-646d-4ada-8ea0-3722c139745a roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
77132025-04-16T04:07:43.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = ade61282-646d-4ada-8ea0-3722c139745a resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
77142025-04-16T04:07:43.560ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3126 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = ade61282-646d-4ada-8ea0-3722c139745a response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?sort_by=id_ascending
77152025-04-16T04:07:43.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = dd117ab4-6166-4ed5-b654-9f33853204e3 roles = RoleSet { roles: {} } uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
77162025-04-16T04:07:43.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = dd117ab4-6166-4ed5-b654-9f33853204e3 resource = Database result = Ok(()) uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
77172025-04-16T04:07:43.562ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2063 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60491 req_id = dd117ab4-6166-4ed5-b654-9f33853204e3 response_code = 200 uri = /metrics/collectors/0efaecbf-c4c4-456a-96b6-2cd10690be0f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTJhMjM1ZC1iNjJkLTQxNjgtYTM2YS02ODcxNzQ3NjFiZjMifX0%3D
77182025-04-16T04:07:43.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 371b76e2-efe0-497f-9be6-8fb9e34b554e roles = RoleSet { roles: {} } uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77192025-04-16T04:07:43.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 371b76e2-efe0-497f-9be6-8fb9e34b554e resource = Database result = Ok(()) uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3
77202025-04-16T04:07:43.610ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2000 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:42814 req_id = 371b76e2-efe0-497f-9be6-8fb9e34b554e response_code = 200 uri = /probes/fe2a235d-b62d-4168-a36a-687174761bf3