Max level shown:
12024-08-28T02:15:02.922ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82514d88-ea8c-4186-950b-9075c7641cf6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
22024-08-28T02:15:02.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82514d88-ea8c-4186-950b-9075c7641cf6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
32024-08-28T02:15:02.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2653 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82514d88-ea8c-4186-950b-9075c7641cf6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
42024-08-28T02:15:03.948ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3c13886c-f175-4b55-94ae-3edbf16a8b0c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
52024-08-28T02:15:03.949ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3c13886c-f175-4b55-94ae-3edbf16a8b0c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
62024-08-28T02:15:03.951ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2638 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3c13886c-f175-4b55-94ae-3edbf16a8b0c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
72024-08-28T02:15:04.971ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 17e356d5-937d-4911-bd54-dc486178a5a3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
82024-08-28T02:15:04.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 17e356d5-937d-4911-bd54-dc486178a5a3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
92024-08-28T02:15:04.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2060 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 17e356d5-937d-4911-bd54-dc486178a5a3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
102024-08-28T02:15:05.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2240 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = c95fd8ff-de81-4ed1-b06e-a2b10b93e572 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
112024-08-28T02:15:05.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db788194-8868-4794-b794-4f46b718e3e4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
122024-08-28T02:15:05.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db788194-8868-4794-b794-4f46b718e3e4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
132024-08-28T02:15:05.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1846 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db788194-8868-4794-b794-4f46b718e3e4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
142024-08-28T02:15:07.013ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ebf852f-a7f3-466e-a0b6-05d3090a4fb9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
152024-08-28T02:15:07.013ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ebf852f-a7f3-466e-a0b6-05d3090a4fb9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
162024-08-28T02:15:07.015ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1821 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ebf852f-a7f3-466e-a0b6-05d3090a4fb9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
172024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
182024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
192024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
202024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
212024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
222024-08-28T02:15:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
232024-08-28T02:15:07.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242024-08-28T02:15:07.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
252024-08-28T02:15:07.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
262024-08-28T02:15:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
272024-08-28T02:15:07.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
282024-08-28T02:15:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
292024-08-28T02:15:07.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
302024-08-28T02:15:07.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
312024-08-28T02:15:07.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
322024-08-28T02:15:07.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
332024-08-28T02:15:07.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
342024-08-28T02:15:07.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
352024-08-28T02:15:07.163ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
362024-08-28T02:15:07.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.958717ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
372024-08-28T02:15:07.173ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.330273ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
382024-08-28T02:15:08.035ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ed4a8f9-7ac8-45a3-b279-1864c9f9689c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
392024-08-28T02:15:08.035ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ed4a8f9-7ac8-45a3-b279-1864c9f9689c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
402024-08-28T02:15:08.037ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2308 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ed4a8f9-7ac8-45a3-b279-1864c9f9689c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
412024-08-28T02:15:09.057ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0f11929-1928-4165-b935-64c058132903 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
422024-08-28T02:15:09.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0f11929-1928-4165-b935-64c058132903 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
432024-08-28T02:15:09.059ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2630 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0f11929-1928-4165-b935-64c058132903 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
442024-08-28T02:15:10.079ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 34f5645c-c450-4914-a78f-922a476c4aff roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
452024-08-28T02:15:10.079ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 34f5645c-c450-4914-a78f-922a476c4aff resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
462024-08-28T02:15:10.081ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 34f5645c-c450-4914-a78f-922a476c4aff response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
472024-08-28T02:15:11.100ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9f38d34b-1988-466b-bee2-4ffd6d5dcba7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
482024-08-28T02:15:11.100ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9f38d34b-1988-466b-bee2-4ffd6d5dcba7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
492024-08-28T02:15:11.102ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2541 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9f38d34b-1988-466b-bee2-4ffd6d5dcba7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
502024-08-28T02:15:12.121ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27bf768d-efce-4cb5-94ba-4258115ff164 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
512024-08-28T02:15:12.121ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27bf768d-efce-4cb5-94ba-4258115ff164 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
522024-08-28T02:15:12.123ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2799 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27bf768d-efce-4cb5-94ba-4258115ff164 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
532024-08-28T02:15:13.142ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 98d18748-4970-4001-848c-bf1067048f5d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
542024-08-28T02:15:13.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 98d18748-4970-4001-848c-bf1067048f5d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
552024-08-28T02:15:13.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2836 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 98d18748-4970-4001-848c-bf1067048f5d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
562024-08-28T02:15:14.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db7b2b27-a4e4-4533-bc12-70f95ca67571 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
572024-08-28T02:15:14.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db7b2b27-a4e4-4533-bc12-70f95ca67571 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
582024-08-28T02:15:14.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db7b2b27-a4e4-4533-bc12-70f95ca67571 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
592024-08-28T02:15:15.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7a6ce57-5599-44f3-8458-b1fdd63de7b4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
602024-08-28T02:15:15.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7a6ce57-5599-44f3-8458-b1fdd63de7b4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
612024-08-28T02:15:15.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1969 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7a6ce57-5599-44f3-8458-b1fdd63de7b4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
622024-08-28T02:15:15.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1449 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 56a7b278-560a-4b7d-88b2-44b61195375c response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
632024-08-28T02:15:16.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e858a39d-26bb-4baa-a6f1-177e4187c7fb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
642024-08-28T02:15:16.206ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e858a39d-26bb-4baa-a6f1-177e4187c7fb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
652024-08-28T02:15:16.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1725 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e858a39d-26bb-4baa-a6f1-177e4187c7fb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
662024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
672024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
682024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
692024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
702024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
712024-08-28T02:15:17.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
722024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
732024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
742024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
752024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
762024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
772024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
782024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
792024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
802024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
812024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
822024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
832024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
842024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
852024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
862024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
872024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
882024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
892024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
902024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
912024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
922024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
932024-08-28T02:15:17.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
942024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
952024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
962024-08-28T02:15:17.145ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758
972024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.841\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758"}
982024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
992024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1002024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
1012024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
1022024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
1032024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1042024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1052024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1062024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
1072024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1082024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
1092024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1102024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
1112024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1122024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
1132024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
1142024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1152024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
1162024-08-28T02:15:17.145ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
1172024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1182024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
1192024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1202024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
1212024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1222024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
1232024-08-28T02:15:17.145ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-28 02:05:17.145535452 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
1242024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
1252024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1262024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1272024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1282024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1292024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1302024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1312024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1322024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1332024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1342024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1352024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1362024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1372024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1382024-08-28T02:15:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1392024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1402024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1412024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1422024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
1432024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1442024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
1452024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1462024-08-28T02:15:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("01868911-e8e4-4a77-b105-b83672ce13b2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c854ff-da18-4c2a-9d2f-7e15248c76a8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1125, tv_nsec: 304375718 } })
1472024-08-28T02:15:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1482024-08-28T02:15:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6926ac43-4e2e-4378-9bce-42019ec72a30.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1125, tv_nsec: 305126702 } })
1492024-08-28T02:15:17.147ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1502024-08-28T02:15:17.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.234375ms iteration = 6 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1512024-08-28T02:15:17.148ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1522024-08-28T02:15:17.148ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.05941ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1532024-08-28T02:15:17.149ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542024-08-28T02:15:17.149ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1552024-08-28T02:15:17.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1562024-08-28T02:15:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.647862ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1572024-08-28T02:15:17.150ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1125, tv_nsec: 308790835 } })
1582024-08-28T02:15:17.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
1592024-08-28T02:15:17.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1602024-08-28T02:15:17.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1612024-08-28T02:15:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1622024-08-28T02:15:17.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1632024-08-28T02:15:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1642024-08-28T02:15:17.153ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652024-08-28T02:15:17.154ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1662024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1672024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1682024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1692024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1702024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1712024-08-28T02:15:17.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1722024-08-28T02:15:17.158ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1732024-08-28T02:15:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1742024-08-28T02:15:17.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1752024-08-28T02:15:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1762024-08-28T02:15:17.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1772024-08-28T02:15:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1782024-08-28T02:15:17.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 17.345764ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
1792024-08-28T02:15:17.163ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1125, tv_nsec: 321591249 } })
1802024-08-28T02:15:17.163ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-28 02:05:17.145535452 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1812024-08-28T02:15:17.163ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.074879ms iteration = 5 status = {"expiration":"2024-08-28T02:05:17.145535452Z","pruned":[]}
1822024-08-28T02:15:17.164ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.856314ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
1832024-08-28T02:15:17.164ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
1842024-08-28T02:15:17.166ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1852024-08-28T02:15:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1862024-08-28T02:15:17.174ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1872024-08-28T02:15:17.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1882024-08-28T02:15:17.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1125, tv_nsec: 333982265 } })
1892024-08-28T02:15:17.175ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1902024-08-28T02:15:17.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1912024-08-28T02:15:17.176ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
1922024-08-28T02:15:17.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1932024-08-28T02:15:17.179ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1942024-08-28T02:15:17.179ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1952024-08-28T02:15:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1962024-08-28T02:15:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1972024-08-28T02:15:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1982024-08-28T02:15:17.180ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1992024-08-28T02:15:17.180ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2002024-08-28T02:15:17.180ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2012024-08-28T02:15:17.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2022024-08-28T02:15:17.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2032024-08-28T02:15:17.183ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2042024-08-28T02:15:17.186ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2052024-08-28T02:15:17.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2062024-08-28T02:15:17.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2072024-08-28T02:15:17.187ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2082024-08-28T02:15:17.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2092024-08-28T02:15:17.189ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 43.831151ms iteration = 9 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
2102024-08-28T02:15:17.191ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
2112024-08-28T02:15:17.192ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
2122024-08-28T02:15:17.192ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 46.777579ms iteration = 5 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
2132024-08-28T02:15:17.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2142024-08-28T02:15:17.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2152024-08-28T02:15:17.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
2162024-08-28T02:15:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2172024-08-28T02:15:17.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2182024-08-28T02:15:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2192024-08-28T02:15:17.201ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.593704ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2202024-08-28T02:15:17.202ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2212024-08-28T02:15:17.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2222024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2232024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2242024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2252024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2262024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2272024-08-28T02:15:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2282024-08-28T02:15:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2292024-08-28T02:15:17.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2302024-08-28T02:15:17.205ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
2312024-08-28T02:15:17.205ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2322024-08-28T02:15:17.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2332024-08-28T02:15:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2342024-08-28T02:15:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2352024-08-28T02:15:17.207ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2362024-08-28T02:15:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2372024-08-28T02:15:17.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2382024-08-28T02:15:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2392024-08-28T02:15:17.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2402024-08-28T02:15:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2412024-08-28T02:15:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2422024-08-28T02:15:17.209ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2432024-08-28T02:15:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2442024-08-28T02:15:17.223ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2452024-08-28T02:15:17.223ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2462024-08-28T02:15:17.223ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2472024-08-28T02:15:17.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2482024-08-28T02:15:17.227ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6f727237-ebef-4e5a-951d-0d3214cfc348 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
2492024-08-28T02:15:17.227ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6f727237-ebef-4e5a-951d-0d3214cfc348 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
2502024-08-28T02:15:17.243ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2512024-08-28T02:15:17.243ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2522024-08-28T02:15:17.243ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2532024-08-28T02:15:17.243ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2542024-08-28T02:15:17.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2552024-08-28T02:15:17.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 6f10e5f0-a5eb-40da-af11-1805b3778318 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
2562024-08-28T02:15:17.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 6f10e5f0-a5eb-40da-af11-1805b3778318 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
2572024-08-28T02:15:17.274ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2582024-08-28T02:15:17.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2592024-08-28T02:15:17.290ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
2602024-08-28T02:15:17.290ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2612024-08-28T02:15:17.291ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "be11cb4f-dbf5-444b-9fd2-889d0f013d0b", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
2622024-08-28T02:15:17.291ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2632024-08-28T02:15:17.293ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2642024-08-28T02:15:17.295ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2652024-08-28T02:15:17.296ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2662024-08-28T02:15:17.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2672024-08-28T02:15:17.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2682024-08-28T02:15:17.341ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
2692024-08-28T02:15:17.341ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2702024-08-28T02:15:17.341ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "871864ce-845a-4f3f-97d1-2c2023d430b0", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
2712024-08-28T02:15:17.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2722024-08-28T02:15:17.343ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2732024-08-28T02:15:17.351ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
2742024-08-28T02:15:17.351ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2752024-08-28T02:15:17.351ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a519d4dd-273c-4deb-8873-b3bb555d8873", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
2762024-08-28T02:15:17.351ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2772024-08-28T02:15:17.353ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
2782024-08-28T02:15:17.360ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2792024-08-28T02:15:17.361ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bcecd944-9171-494f-9b87-f1b86e962953", "content-length": "372", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
2802024-08-28T02:15:17.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2812024-08-28T02:15:17.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2822024-08-28T02:15:17.363ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 218.149892ms iteration = 9 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
2832024-08-28T02:15:17.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2842024-08-28T02:15:17.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2852024-08-28T02:15:17.365ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2862024-08-28T02:15:17.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2872024-08-28T02:15:17.366ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2882024-08-28T02:15:17.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2892024-08-28T02:15:17.367ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2902024-08-28T02:15:17.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 140424 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6f727237-ebef-4e5a-951d-0d3214cfc348 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
2912024-08-28T02:15:17.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2922024-08-28T02:15:17.371ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2932024-08-28T02:15:17.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2942024-08-28T02:15:17.372ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2952024-08-28T02:15:17.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2962024-08-28T02:15:17.374ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2972024-08-28T02:15:17.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2982024-08-28T02:15:17.377ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 127574 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 6f10e5f0-a5eb-40da-af11-1805b3778318 response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
2992024-08-28T02:15:17.387ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3002024-08-28T02:15:17.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3012024-08-28T02:15:17.390ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3022024-08-28T02:15:17.391ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "401f8b33-86c7-450e-a663-aa56aeec2ed4", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
3032024-08-28T02:15:17.393ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3042024-08-28T02:15:17.393ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 248.120207ms iteration = 10 status = {}
3052024-08-28T02:15:17.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3062024-08-28T02:15:17.401ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3072024-08-28T02:15:17.403ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3082024-08-28T02:15:17.403ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3092024-08-28T02:15:17.403ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 257.768769ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3102024-08-28T02:15:17.403ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
3112024-08-28T02:15:17.403ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3122024-08-28T02:15:17.403ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 257.82708ms iteration = 9 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
3132024-08-28T02:15:17.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3142024-08-28T02:15:17.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3152024-08-28T02:15:17.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3162024-08-28T02:15:17.407ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
3172024-08-28T02:15:17.407ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3182024-08-28T02:15:17.407ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-28 02:11:34.295733 UTC dns_group = internal generation = 1
3192024-08-28T02:15:17.407ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 261.759598ms iteration = 6 status = {"generation":1}
3202024-08-28T02:15:17.408ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3212024-08-28T02:15:17.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3222024-08-28T02:15:17.409ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3232024-08-28T02:15:17.410ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3242024-08-28T02:15:17.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3252024-08-28T02:15:17.410ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3262024-08-28T02:15:17.411ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3272024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3282024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 0 version = 2
3292024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3302024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3312024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3322024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3332024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3342024-08-28T02:15:17.412ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3352024-08-28T02:15:17.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3362024-08-28T02:15:17.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3372024-08-28T02:15:17.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3382024-08-28T02:15:17.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3392024-08-28T02:15:17.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3402024-08-28T02:15:17.414ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3412024-08-28T02:15:17.415ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3422024-08-28T02:15:17.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3432024-08-28T02:15:17.417ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3442024-08-28T02:15:17.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452024-08-28T02:15:17.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3462024-08-28T02:15:17.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
3472024-08-28T02:15:17.418ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3482024-08-28T02:15:17.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3492024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
3502024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3512024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3522024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 1 version = 2
3532024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3542024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-28 02:11:35.148647 UTC dns_group = external generation = 2
3552024-08-28T02:15:17.421ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 276.745885ms iteration = 6 status = {"generation":2}
3562024-08-28T02:15:17.422ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 277.737645ms iteration = 9 status = 0
3572024-08-28T02:15:17.433ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
3582024-08-28T02:15:17.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 288.010699ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
3592024-08-28T02:15:17.434ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3602024-08-28T02:15:17.434ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 289.330646ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3612024-08-28T02:15:17.435ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
3622024-08-28T02:15:17.435ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 289.811785ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
3632024-08-28T02:15:17.451ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-28T02:11:15.536604Z, time_modified: 2024-08-28T02:11:15.536604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }) }
3642024-08-28T02:15:17.451ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 306.653479ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]}},"default_endpoint":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3652024-08-28T02:15:17.455ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3662024-08-28T02:15:17.456ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1f7a26f7-d80a-4f25-b35e-89d5c629763a", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3672024-08-28T02:15:17.456ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3682024-08-28T02:15:17.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2ef5bb96-b5f8-4820-8782-7f93096509b4", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3692024-08-28T02:15:17.457ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3702024-08-28T02:15:17.459ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3712024-08-28T02:15:17.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 314.093397ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3722024-08-28T02:15:17.484ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3732024-08-28T02:15:17.485ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "419a6d0b-4781-447e-b6d7-d0f138b4d220", "content-length": "1", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
3742024-08-28T02:15:17.486ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3752024-08-28T02:15:17.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3762024-08-28T02:15:17.488ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 342.901229ms iteration = 9 status = null
3772024-08-28T02:15:17.502ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 356.763584ms iteration = 12 status = {"physical_disks_added":0}
3782024-08-28T02:15:17.567ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 421.913488ms iteration = 9 status = {}
3792024-08-28T02:15:17.568ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3802024-08-28T02:15:17.582ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3812024-08-28T02:15:17.583ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4c5c8593-69de-424d-9e50-71f6fbc817bf", "content-length": "691", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
3822024-08-28T02:15:17.583ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3832024-08-28T02:15:17.583ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3842024-08-28T02:15:17.583ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3852024-08-28T02:15:17.585ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 440.003107ms iteration = 9 status = {}
3862024-08-28T02:15:17.614ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3872024-08-28T02:15:17.614ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "798d9e32-b8b9-40d8-8070-fbf79be25c45", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3882024-08-28T02:15:17.614ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3892024-08-28T02:15:17.615ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "17737f14-8968-4a0a-9107-f6a9724d5b21", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3902024-08-28T02:15:17.615ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3912024-08-28T02:15:17.616ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6070451e-02a4-43bf-8065-e09ff24ae538", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3922024-08-28T02:15:17.616ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3932024-08-28T02:15:17.618ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
3942024-08-28T02:15:17.618ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 472.741247ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3952024-08-28T02:15:17.626ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3962024-08-28T02:15:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3972024-08-28T02:15:17.629ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
3982024-08-28T02:15:17.629ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3992024-08-28T02:15:17.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4002024-08-28T02:15:17.640ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4012024-08-28T02:15:17.640ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "888d4ebe-ed8c-4d06-9ace-d1748a3313e2", "content-length": "121", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
4022024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4032024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
4042024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4052024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4062024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
4072024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
4082024-08-28T02:15:17.640ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
4092024-08-28T02:15:17.640ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4102024-08-28T02:15:17.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4112024-08-28T02:15:17.643ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4122024-08-28T02:15:17.643ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5e9d9f81-4779-45f3-9340-0709afbd123e", "content-length": "2", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
4132024-08-28T02:15:17.643ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4142024-08-28T02:15:17.643ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "222b3b44-009e-4779-9971-16971af56d38", "content-length": "51", "date": "Wed, 28 Aug 2024 02:15:16 GMT"} })
4152024-08-28T02:15:17.643ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
4162024-08-28T02:15:17.643ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4172024-08-28T02:15:17.729ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8a2eec1c-65b0-4e72-a1f8-6674b0715077", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
4182024-08-28T02:15:17.730ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
4192024-08-28T02:15:17.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4202024-08-28T02:15:17.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "608f8767-8ef2-4aa5-9420-6ac6229ebb06", "content-length": "667", "date": "Wed, 28 Aug 2024 02:15:17 GMT"} })
4212024-08-28T02:15:17.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4222024-08-28T02:15:17.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4232024-08-28T02:15:17.734ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4242024-08-28T02:15:17.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4252024-08-28T02:15:17.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4262024-08-28T02:15:17.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4272024-08-28T02:15:17.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4282024-08-28T02:15:17.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4292024-08-28T02:15:17.746ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 600.840591ms iteration = 9 status = {}
4302024-08-28T02:15:18.391ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5c449c-edf1-4d4b-90d9-2d0a12862816 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4312024-08-28T02:15:18.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5c449c-edf1-4d4b-90d9-2d0a12862816 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4322024-08-28T02:15:18.394ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2673 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5c449c-edf1-4d4b-90d9-2d0a12862816 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4332024-08-28T02:15:19.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4414928a-c937-434d-a4a7-a1f3f3ebedc6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4342024-08-28T02:15:19.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4414928a-c937-434d-a4a7-a1f3f3ebedc6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4352024-08-28T02:15:19.417ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2789 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4414928a-c937-434d-a4a7-a1f3f3ebedc6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4362024-08-28T02:15:20.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7a0332e0-5165-45c1-b0da-e64232f4e55a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4372024-08-28T02:15:20.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7a0332e0-5165-45c1-b0da-e64232f4e55a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4382024-08-28T02:15:20.437ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3129 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7a0332e0-5165-45c1-b0da-e64232f4e55a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4392024-08-28T02:15:21.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fca8ef7d-bd7e-4dec-be40-8fbd6dc45209 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4402024-08-28T02:15:21.457ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fca8ef7d-bd7e-4dec-be40-8fbd6dc45209 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4412024-08-28T02:15:21.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2991 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fca8ef7d-bd7e-4dec-be40-8fbd6dc45209 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4422024-08-28T02:15:22.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71de202c-76e1-4024-af53-d7ee692cd5c4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4432024-08-28T02:15:22.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71de202c-76e1-4024-af53-d7ee692cd5c4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4442024-08-28T02:15:22.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2767 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71de202c-76e1-4024-af53-d7ee692cd5c4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4452024-08-28T02:15:23.502ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1fcf4c37-a646-4238-a53b-6dc4b012f2f6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4462024-08-28T02:15:23.502ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1fcf4c37-a646-4238-a53b-6dc4b012f2f6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4472024-08-28T02:15:23.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1fcf4c37-a646-4238-a53b-6dc4b012f2f6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4482024-08-28T02:15:24.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 70ea7a56-100b-4f31-ae00-fbf9faf23f3e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4492024-08-28T02:15:24.523ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 70ea7a56-100b-4f31-ae00-fbf9faf23f3e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4502024-08-28T02:15:24.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2275 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 70ea7a56-100b-4f31-ae00-fbf9faf23f3e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4512024-08-28T02:15:25.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ac352c1-6bce-4367-9016-853a07ae1f82 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4522024-08-28T02:15:25.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ac352c1-6bce-4367-9016-853a07ae1f82 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4532024-08-28T02:15:25.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2017 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9ac352c1-6bce-4367-9016-853a07ae1f82 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4542024-08-28T02:15:25.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1511 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 74317ec0-a716-496a-8554-4a098a8a87eb response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
4552024-08-28T02:15:26.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2decf403-fe80-4f11-8a3c-050af1ae2bf5 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4562024-08-28T02:15:26.565ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2decf403-fe80-4f11-8a3c-050af1ae2bf5 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4572024-08-28T02:15:26.567ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2034 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2decf403-fe80-4f11-8a3c-050af1ae2bf5 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4582024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
4592024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4602024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
4612024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4622024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4632024-08-28T02:15:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4642024-08-28T02:15:27.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4652024-08-28T02:15:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4662024-08-28T02:15:27.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4672024-08-28T02:15:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4682024-08-28T02:15:27.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4692024-08-28T02:15:27.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4702024-08-28T02:15:27.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4712024-08-28T02:15:27.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
4722024-08-28T02:15:27.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4732024-08-28T02:15:27.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4742024-08-28T02:15:27.166ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
4752024-08-28T02:15:27.166ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.806094ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
4762024-08-28T02:15:27.173ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4772024-08-28T02:15:27.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4782024-08-28T02:15:27.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 54.621794ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4792024-08-28T02:15:27.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b92d8530-deaa-49f2-ae19-8b7294c47b5c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4802024-08-28T02:15:27.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b92d8530-deaa-49f2-ae19-8b7294c47b5c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4812024-08-28T02:15:27.588ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2155 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b92d8530-deaa-49f2-ae19-8b7294c47b5c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4822024-08-28T02:15:28.607ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ad71460-2cfb-4255-8bc3-6a1f4dfce099 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4832024-08-28T02:15:28.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ad71460-2cfb-4255-8bc3-6a1f4dfce099 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4842024-08-28T02:15:28.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1929 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ad71460-2cfb-4255-8bc3-6a1f4dfce099 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4852024-08-28T02:15:29.627ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e9c4273f-4c98-47d9-bde0-c7e97644739f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4862024-08-28T02:15:29.628ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e9c4273f-4c98-47d9-bde0-c7e97644739f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4872024-08-28T02:15:29.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2149 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e9c4273f-4c98-47d9-bde0-c7e97644739f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4882024-08-28T02:15:30.650ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bf023f65-b0ea-4d25-9ae2-25548b86efef roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4892024-08-28T02:15:30.650ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bf023f65-b0ea-4d25-9ae2-25548b86efef resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4902024-08-28T02:15:30.652ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2045 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bf023f65-b0ea-4d25-9ae2-25548b86efef response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4912024-08-28T02:15:31.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c79500-c48b-4df6-b70a-f1c10b399a42 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4922024-08-28T02:15:31.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c79500-c48b-4df6-b70a-f1c10b399a42 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4932024-08-28T02:15:31.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c79500-c48b-4df6-b70a-f1c10b399a42 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4942024-08-28T02:15:32.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa169557-d120-475d-b1d3-4dc2ea562b96 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4952024-08-28T02:15:32.693ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa169557-d120-475d-b1d3-4dc2ea562b96 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4962024-08-28T02:15:32.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1715 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa169557-d120-475d-b1d3-4dc2ea562b96 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4972024-08-28T02:15:33.715ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 47c9b5a4-85b5-4021-95ca-003beeaeb809 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4982024-08-28T02:15:33.715ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 47c9b5a4-85b5-4021-95ca-003beeaeb809 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
4992024-08-28T02:15:33.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2524 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 47c9b5a4-85b5-4021-95ca-003beeaeb809 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5002024-08-28T02:15:34.735ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b8a5571c-2095-4a4e-b7e4-9296f275d37d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5012024-08-28T02:15:34.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b8a5571c-2095-4a4e-b7e4-9296f275d37d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5022024-08-28T02:15:34.737ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2125 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b8a5571c-2095-4a4e-b7e4-9296f275d37d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5032024-08-28T02:15:35.758ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a854e7a-2e26-4a3f-832b-088454f5cb76 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5042024-08-28T02:15:35.758ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a854e7a-2e26-4a3f-832b-088454f5cb76 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5052024-08-28T02:15:35.760ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2192 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a854e7a-2e26-4a3f-832b-088454f5cb76 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5062024-08-28T02:15:35.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1664 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = b19ec039-683a-4688-b091-449ab5cdf899 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
5072024-08-28T02:15:36.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aaa91861-6a70-4ffb-afe9-46dcbf8cc179 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5082024-08-28T02:15:36.779ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aaa91861-6a70-4ffb-afe9-46dcbf8cc179 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5092024-08-28T02:15:36.781ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2675 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aaa91861-6a70-4ffb-afe9-46dcbf8cc179 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5102024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
5112024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5122024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
5132024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5142024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5152024-08-28T02:15:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5162024-08-28T02:15:37.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5172024-08-28T02:15:37.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5182024-08-28T02:15:37.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5192024-08-28T02:15:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5202024-08-28T02:15:37.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5212024-08-28T02:15:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5222024-08-28T02:15:37.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5232024-08-28T02:15:37.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5242024-08-28T02:15:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5252024-08-28T02:15:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
5262024-08-28T02:15:37.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5272024-08-28T02:15:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5282024-08-28T02:15:37.165ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
5292024-08-28T02:15:37.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.384475ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
5302024-08-28T02:15:37.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.269401ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5312024-08-28T02:15:37.800ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439294e7-a5a0-4509-9799-053570d8f4ae roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5322024-08-28T02:15:37.800ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439294e7-a5a0-4509-9799-053570d8f4ae resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5332024-08-28T02:15:37.802ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1919 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439294e7-a5a0-4509-9799-053570d8f4ae response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5342024-08-28T02:15:38.821ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45c0f521-3c17-45f6-83bd-c4832fdc4746 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5352024-08-28T02:15:38.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45c0f521-3c17-45f6-83bd-c4832fdc4746 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5362024-08-28T02:15:38.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2863 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45c0f521-3c17-45f6-83bd-c4832fdc4746 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5372024-08-28T02:15:39.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86348ac6-6617-40a3-b391-8e294366ae06 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5382024-08-28T02:15:39.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86348ac6-6617-40a3-b391-8e294366ae06 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5392024-08-28T02:15:39.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86348ac6-6617-40a3-b391-8e294366ae06 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5402024-08-28T02:15:40.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 40aa05a2-4f5a-43df-bd94-9a747e5ec21d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5412024-08-28T02:15:40.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 40aa05a2-4f5a-43df-bd94-9a747e5ec21d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5422024-08-28T02:15:40.867ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 40aa05a2-4f5a-43df-bd94-9a747e5ec21d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5432024-08-28T02:15:41.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1107c185-cc07-46bc-97cf-cfb4e32945a7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5442024-08-28T02:15:41.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1107c185-cc07-46bc-97cf-cfb4e32945a7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5452024-08-28T02:15:41.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2390 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1107c185-cc07-46bc-97cf-cfb4e32945a7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5462024-08-28T02:15:42.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 668bc413-083e-488b-b000-bc858dae742d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5472024-08-28T02:15:42.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 668bc413-083e-488b-b000-bc858dae742d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5482024-08-28T02:15:42.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1814 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 668bc413-083e-488b-b000-bc858dae742d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5492024-08-28T02:15:43.928ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84793021-ee5e-4c80-a4a1-d3e1095bae74 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5502024-08-28T02:15:43.929ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84793021-ee5e-4c80-a4a1-d3e1095bae74 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5512024-08-28T02:15:43.930ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2197 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84793021-ee5e-4c80-a4a1-d3e1095bae74 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5522024-08-28T02:15:44.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185ea9d6-9909-48e3-a20a-e6300bcd4321 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5532024-08-28T02:15:44.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185ea9d6-9909-48e3-a20a-e6300bcd4321 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5542024-08-28T02:15:44.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1990 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185ea9d6-9909-48e3-a20a-e6300bcd4321 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5552024-08-28T02:15:45.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1939 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 64110445-d1fe-43d2-91d4-0dbc2d0b4d7f response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
5562024-08-28T02:15:45.970ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = de486fa5-6d82-4be9-925c-4597e62df2e9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5572024-08-28T02:15:45.971ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = de486fa5-6d82-4be9-925c-4597e62df2e9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5582024-08-28T02:15:45.972ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1691 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = de486fa5-6d82-4be9-925c-4597e62df2e9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5592024-08-28T02:15:46.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ede52bdc-04fd-4099-9ffc-57e419c5cf74 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5602024-08-28T02:15:46.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ede52bdc-04fd-4099-9ffc-57e419c5cf74 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5612024-08-28T02:15:46.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1998 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ede52bdc-04fd-4099-9ffc-57e419c5cf74 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
5622024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
5632024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5642024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
5652024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5662024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
5672024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5682024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
5692024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5702024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5712024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
5722024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5732024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
5742024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
5752024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
5762024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5772024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5782024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
5792024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
5802024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5812024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
5822024-08-28T02:15:47.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
5832024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5842024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5852024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
5862024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
5872024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5882024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5892024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
5902024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5912024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
5922024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5932024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
5942024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5952024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5962024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5972024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5982024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5992024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6002024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6012024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6022024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6032024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6042024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6052024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6062024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6072024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6082024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6092024-08-28T02:15:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6102024-08-28T02:15:47.147ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.4782ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6112024-08-28T02:15:47.148ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6122024-08-28T02:15:47.148ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6132024-08-28T02:15:47.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6142024-08-28T02:15:47.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6152024-08-28T02:15:47.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6162024-08-28T02:15:47.150ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6172024-08-28T02:15:47.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6182024-08-28T02:15:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6192024-08-28T02:15:47.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6202024-08-28T02:15:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6212024-08-28T02:15:47.152ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6222024-08-28T02:15:47.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6232024-08-28T02:15:47.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6242024-08-28T02:15:47.154ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6252024-08-28T02:15:47.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6262024-08-28T02:15:47.154ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1155, tv_nsec: 313013389 } })
6272024-08-28T02:15:47.155ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
6282024-08-28T02:15:47.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6292024-08-28T02:15:47.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6302024-08-28T02:15:47.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6312024-08-28T02:15:47.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6322024-08-28T02:15:47.157ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1155, tv_nsec: 315631971 } })
6332024-08-28T02:15:47.158ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
6342024-08-28T02:15:47.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6352024-08-28T02:15:47.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6362024-08-28T02:15:47.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6372024-08-28T02:15:47.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6382024-08-28T02:15:47.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6392024-08-28T02:15:47.163ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1155, tv_nsec: 321107480 } })
6402024-08-28T02:15:47.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6412024-08-28T02:15:47.164ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
6422024-08-28T02:15:47.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6432024-08-28T02:15:47.184ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 38.948974ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
6442024-08-28T02:15:47.185ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6452024-08-28T02:15:47.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6462024-08-28T02:15:47.185ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
6472024-08-28T02:15:47.185ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6482024-08-28T02:15:47.185ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6492024-08-28T02:15:47.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6502024-08-28T02:15:47.186ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6512024-08-28T02:15:47.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6522024-08-28T02:15:47.187ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6532024-08-28T02:15:47.188ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6542024-08-28T02:15:47.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6552024-08-28T02:15:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6562024-08-28T02:15:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6572024-08-28T02:15:47.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6582024-08-28T02:15:47.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6592024-08-28T02:15:47.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6602024-08-28T02:15:47.191ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 46.709507ms iteration = 10 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
6612024-08-28T02:15:47.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6622024-08-28T02:15:47.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6632024-08-28T02:15:47.192ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6642024-08-28T02:15:47.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6652024-08-28T02:15:47.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6662024-08-28T02:15:47.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.34201ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6672024-08-28T02:15:47.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
6682024-08-28T02:15:47.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6692024-08-28T02:15:47.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6702024-08-28T02:15:47.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6712024-08-28T02:15:47.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6722024-08-28T02:15:47.197ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6732024-08-28T02:15:47.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6742024-08-28T02:15:47.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6752024-08-28T02:15:47.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6762024-08-28T02:15:47.199ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6772024-08-28T02:15:47.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6782024-08-28T02:15:47.212ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
6792024-08-28T02:15:47.213ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.894238ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
6802024-08-28T02:15:47.221ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6812024-08-28T02:15:47.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6822024-08-28T02:15:47.223ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
6832024-08-28T02:15:47.223ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6842024-08-28T02:15:47.223ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 78.440098ms iteration = 10 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
6852024-08-28T02:15:47.223ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 78.735553ms iteration = 10 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6862024-08-28T02:15:47.228ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6872024-08-28T02:15:47.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
6882024-08-28T02:15:47.231ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6892024-08-28T02:15:47.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6902024-08-28T02:15:47.239ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6912024-08-28T02:15:47.239ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 94.16881ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6922024-08-28T02:15:47.244ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
6932024-08-28T02:15:47.244ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6942024-08-28T02:15:47.244ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ee9625d-e370-46ff-babb-51a996d6325b", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
6952024-08-28T02:15:47.244ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6962024-08-28T02:15:47.245ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6972024-08-28T02:15:47.251ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
6982024-08-28T02:15:47.251ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6992024-08-28T02:15:47.252ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a4e8b522-b7f7-40d0-9a93-4da4b6217959", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
7002024-08-28T02:15:47.252ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7012024-08-28T02:15:47.254ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7022024-08-28T02:15:47.261ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7032024-08-28T02:15:47.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7042024-08-28T02:15:47.296ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 151.179472ms iteration = 13 status = {"physical_disks_added":0}
7052024-08-28T02:15:47.318ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
7062024-08-28T02:15:47.318ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7072024-08-28T02:15:47.318ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "314b2405-3d30-4fb5-b7b1-0fb27d963965", "content-length": "26", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
7082024-08-28T02:15:47.318ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7092024-08-28T02:15:47.320ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
7102024-08-28T02:15:47.379ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7112024-08-28T02:15:47.379ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c17e430-773c-4ba8-89fa-005aa941eb5e", "content-length": "372", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7122024-08-28T02:15:47.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7132024-08-28T02:15:47.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7142024-08-28T02:15:47.381ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 235.996707ms iteration = 10 status = 0
7152024-08-28T02:15:47.382ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7162024-08-28T02:15:47.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7172024-08-28T02:15:47.385ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7182024-08-28T02:15:47.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7192024-08-28T02:15:47.388ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7202024-08-28T02:15:47.388ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8c5017b7-6cf1-4c3d-bad3-7428a3bfa09d", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7212024-08-28T02:15:47.390ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 245.715009ms iteration = 11 status = {}
7222024-08-28T02:15:47.406ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
7232024-08-28T02:15:47.406ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 261.582715ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
7242024-08-28T02:15:47.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = e7e35af6-f90a-41af-93f5-c361bd7aaf70 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
7252024-08-28T02:15:47.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = e7e35af6-f90a-41af-93f5-c361bd7aaf70 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
7262024-08-28T02:15:47.413ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2916 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = e7e35af6-f90a-41af-93f5-c361bd7aaf70 response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
7272024-08-28T02:15:47.413ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7282024-08-28T02:15:47.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7292024-08-28T02:15:47.415ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302024-08-28T02:15:47.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7312024-08-28T02:15:47.417ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7322024-08-28T02:15:47.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7332024-08-28T02:15:47.419ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7342024-08-28T02:15:47.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7352024-08-28T02:15:47.421ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7362024-08-28T02:15:47.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7372024-08-28T02:15:47.452ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7382024-08-28T02:15:47.453ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c17e33ff-a279-44ac-8789-ec8593ada350", "content-length": "1", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7392024-08-28T02:15:47.454ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7402024-08-28T02:15:47.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7412024-08-28T02:15:47.456ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 310.789092ms iteration = 10 status = null
7422024-08-28T02:15:47.466ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 321.525765ms iteration = 10 status = {}
7432024-08-28T02:15:47.533ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7442024-08-28T02:15:47.533ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "80949978-cff6-4432-ab46-fc94ad6d8681", "content-length": "691", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
7452024-08-28T02:15:47.533ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7462024-08-28T02:15:47.533ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7472024-08-28T02:15:47.533ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7482024-08-28T02:15:47.535ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 390.378052ms iteration = 10 status = {}
7492024-08-28T02:15:47.537ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7502024-08-28T02:15:47.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7512024-08-28T02:15:47.540ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
7522024-08-28T02:15:47.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7532024-08-28T02:15:47.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7542024-08-28T02:15:47.550ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7552024-08-28T02:15:47.550ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9669c022-a771-4bca-9731-9632483aac27", "content-length": "121", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7562024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
7572024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
7582024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7592024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7602024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
7612024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
7622024-08-28T02:15:47.550ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
7632024-08-28T02:15:47.550ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7642024-08-28T02:15:47.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7652024-08-28T02:15:47.552ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7662024-08-28T02:15:47.553ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "708f81c8-414d-4139-b11f-e9a41da6591f", "content-length": "2", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7672024-08-28T02:15:47.553ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7682024-08-28T02:15:47.553ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "49296b5a-7758-483e-ae0a-63a6786b268c", "content-length": "51", "date": "Wed, 28 Aug 2024 02:15:46 GMT"} })
7692024-08-28T02:15:47.553ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
7702024-08-28T02:15:47.553ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7712024-08-28T02:15:47.648ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9ff8e5b6-457c-4671-b7a5-651569990c76", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
7722024-08-28T02:15:47.649ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
7732024-08-28T02:15:47.649ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7742024-08-28T02:15:47.649ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "efc21bb8-9a21-4e47-b80c-0bfc0add4ecc", "content-length": "667", "date": "Wed, 28 Aug 2024 02:15:47 GMT"} })
7752024-08-28T02:15:47.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7762024-08-28T02:15:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7772024-08-28T02:15:47.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7782024-08-28T02:15:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7792024-08-28T02:15:47.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7802024-08-28T02:15:47.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7812024-08-28T02:15:47.659ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7822024-08-28T02:15:47.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7832024-08-28T02:15:47.665ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 520.71443ms iteration = 10 status = {}
7842024-08-28T02:15:48.014ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439fece0-9c1d-4348-87f0-441736c3fab5 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7852024-08-28T02:15:48.015ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439fece0-9c1d-4348-87f0-441736c3fab5 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7862024-08-28T02:15:48.016ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2372 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 439fece0-9c1d-4348-87f0-441736c3fab5 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7872024-08-28T02:15:49.037ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 861c92ee-5765-4818-9a7e-bfdc53bfb74f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7882024-08-28T02:15:49.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 861c92ee-5765-4818-9a7e-bfdc53bfb74f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7892024-08-28T02:15:49.039ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2065 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 861c92ee-5765-4818-9a7e-bfdc53bfb74f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7902024-08-28T02:15:50.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12746afe-b5eb-42e3-bed1-84c395d62fe9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7912024-08-28T02:15:50.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12746afe-b5eb-42e3-bed1-84c395d62fe9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7922024-08-28T02:15:50.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12746afe-b5eb-42e3-bed1-84c395d62fe9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7932024-08-28T02:15:51.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d3ed1161-6711-43f4-a6f5-855e1b73e5f2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7942024-08-28T02:15:51.080ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d3ed1161-6711-43f4-a6f5-855e1b73e5f2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7952024-08-28T02:15:51.082ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2727 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d3ed1161-6711-43f4-a6f5-855e1b73e5f2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7962024-08-28T02:15:52.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dda2f190-3d7e-46ab-859b-e49b758c631c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7972024-08-28T02:15:52.101ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dda2f190-3d7e-46ab-859b-e49b758c631c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7982024-08-28T02:15:52.103ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dda2f190-3d7e-46ab-859b-e49b758c631c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
7992024-08-28T02:15:53.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 443fdbb4-3e3f-439c-9aae-9fcaf5b6629a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8002024-08-28T02:15:53.122ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 443fdbb4-3e3f-439c-9aae-9fcaf5b6629a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8012024-08-28T02:15:53.124ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2372 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 443fdbb4-3e3f-439c-9aae-9fcaf5b6629a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8022024-08-28T02:15:54.143ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9fe6beab-2336-4a8b-a64d-955b9c8ddba2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8032024-08-28T02:15:54.143ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9fe6beab-2336-4a8b-a64d-955b9c8ddba2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8042024-08-28T02:15:54.145ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9fe6beab-2336-4a8b-a64d-955b9c8ddba2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8052024-08-28T02:15:55.166ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8a261f05-6df9-4828-97e0-7198a0fb349c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8062024-08-28T02:15:55.166ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8a261f05-6df9-4828-97e0-7198a0fb349c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8072024-08-28T02:15:55.168ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8a261f05-6df9-4828-97e0-7198a0fb349c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8082024-08-28T02:15:55.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1468 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 9f34a6b2-5b62-4e8e-828c-3f1a45974448 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
8092024-08-28T02:15:56.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff8cfe1f-ab2b-4de0-9ee7-06f5d5eec40a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8102024-08-28T02:15:56.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff8cfe1f-ab2b-4de0-9ee7-06f5d5eec40a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8112024-08-28T02:15:56.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff8cfe1f-ab2b-4de0-9ee7-06f5d5eec40a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8122024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
8132024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8142024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
8152024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8162024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8172024-08-28T02:15:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8182024-08-28T02:15:57.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8192024-08-28T02:15:57.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8202024-08-28T02:15:57.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8212024-08-28T02:15:57.148ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.830856ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8222024-08-28T02:15:57.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8232024-08-28T02:15:57.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8242024-08-28T02:15:57.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8252024-08-28T02:15:57.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8262024-08-28T02:15:57.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8272024-08-28T02:15:57.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8282024-08-28T02:15:57.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
8292024-08-28T02:15:57.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8302024-08-28T02:15:57.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8312024-08-28T02:15:57.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
8322024-08-28T02:15:57.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.60155ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
8332024-08-28T02:15:57.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 00c9b8df-2474-45e5-9499-33b64b111a28 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8342024-08-28T02:15:57.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 00c9b8df-2474-45e5-9499-33b64b111a28 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8352024-08-28T02:15:57.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 00c9b8df-2474-45e5-9499-33b64b111a28 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8362024-08-28T02:15:58.230ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6bb2193e-258f-4c79-b9f1-cbfcacb6c1b1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8372024-08-28T02:15:58.230ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6bb2193e-258f-4c79-b9f1-cbfcacb6c1b1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8382024-08-28T02:15:58.232ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2821 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6bb2193e-258f-4c79-b9f1-cbfcacb6c1b1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8392024-08-28T02:15:59.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f6d86089-c6a9-4663-b518-604751a8e012 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8402024-08-28T02:15:59.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f6d86089-c6a9-4663-b518-604751a8e012 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8412024-08-28T02:15:59.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f6d86089-c6a9-4663-b518-604751a8e012 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8422024-08-28T02:16:00.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee75c63d-96df-4af7-b994-fe4b8ecb6a15 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8432024-08-28T02:16:00.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee75c63d-96df-4af7-b994-fe4b8ecb6a15 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8442024-08-28T02:16:00.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2447 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee75c63d-96df-4af7-b994-fe4b8ecb6a15 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8452024-08-28T02:16:01.292ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a685cca0-1c46-48b5-b47b-094a855b69c2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8462024-08-28T02:16:01.292ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a685cca0-1c46-48b5-b47b-094a855b69c2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8472024-08-28T02:16:01.294ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3167 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a685cca0-1c46-48b5-b47b-094a855b69c2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8482024-08-28T02:16:02.312ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37f384c5-0db9-41cb-b7f0-6761359decf3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8492024-08-28T02:16:02.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37f384c5-0db9-41cb-b7f0-6761359decf3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8502024-08-28T02:16:02.314ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1635 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37f384c5-0db9-41cb-b7f0-6761359decf3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8512024-08-28T02:16:03.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ef6b2a7-f61b-4aeb-aa2c-190f2b653941 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8522024-08-28T02:16:03.333ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ef6b2a7-f61b-4aeb-aa2c-190f2b653941 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8532024-08-28T02:16:03.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ef6b2a7-f61b-4aeb-aa2c-190f2b653941 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8542024-08-28T02:16:04.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2aec36bd-c65a-43ca-b1d1-99c83b5152b1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8552024-08-28T02:16:04.355ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2aec36bd-c65a-43ca-b1d1-99c83b5152b1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8562024-08-28T02:16:04.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1948 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2aec36bd-c65a-43ca-b1d1-99c83b5152b1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8572024-08-28T02:16:05.377ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25a505f9-1e8f-4bf5-9d29-13e309225cb3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8582024-08-28T02:16:05.377ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25a505f9-1e8f-4bf5-9d29-13e309225cb3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8592024-08-28T02:16:05.379ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1739 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25a505f9-1e8f-4bf5-9d29-13e309225cb3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8602024-08-28T02:16:05.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1471 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 3ee9f543-cef7-4b73-ae64-2a97f3f09c47 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
8612024-08-28T02:16:06.397ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12bdf89b-2619-4adf-8cb5-ba8f9bc7afe6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8622024-08-28T02:16:06.397ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12bdf89b-2619-4adf-8cb5-ba8f9bc7afe6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8632024-08-28T02:16:06.399ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2053 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12bdf89b-2619-4adf-8cb5-ba8f9bc7afe6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8642024-08-28T02:16:07.144ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
8652024-08-28T02:16:07.144ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8662024-08-28T02:16:07.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
8672024-08-28T02:16:07.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8682024-08-28T02:16:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8692024-08-28T02:16:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8702024-08-28T02:16:07.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8712024-08-28T02:16:07.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8722024-08-28T02:16:07.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8732024-08-28T02:16:07.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8742024-08-28T02:16:07.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8752024-08-28T02:16:07.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8762024-08-28T02:16:07.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8772024-08-28T02:16:07.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8782024-08-28T02:16:07.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8792024-08-28T02:16:07.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
8802024-08-28T02:16:07.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8812024-08-28T02:16:07.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8822024-08-28T02:16:07.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
8832024-08-28T02:16:07.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.910145ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
8842024-08-28T02:16:07.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.357263ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8852024-08-28T02:16:07.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e246d0f-51b6-43e3-a574-687f01dc4956 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8862024-08-28T02:16:07.419ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e246d0f-51b6-43e3-a574-687f01dc4956 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8872024-08-28T02:16:07.421ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2660 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e246d0f-51b6-43e3-a574-687f01dc4956 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8882024-08-28T02:16:08.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 67a38680-c468-43e7-aeb8-3f2150b097db roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8892024-08-28T02:16:08.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 67a38680-c468-43e7-aeb8-3f2150b097db resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8902024-08-28T02:16:08.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1904 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 67a38680-c468-43e7-aeb8-3f2150b097db response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8912024-08-28T02:16:09.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 46aba669-a2c7-462f-b2b6-2cbc129f667d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8922024-08-28T02:16:09.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 46aba669-a2c7-462f-b2b6-2cbc129f667d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8932024-08-28T02:16:09.463ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2060 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 46aba669-a2c7-462f-b2b6-2cbc129f667d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8942024-08-28T02:16:10.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1eb5817f-8173-488c-a08b-1a6eea2ddafe roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8952024-08-28T02:16:10.482ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1eb5817f-8173-488c-a08b-1a6eea2ddafe resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8962024-08-28T02:16:10.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2059 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1eb5817f-8173-488c-a08b-1a6eea2ddafe response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8972024-08-28T02:16:11.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f654c386-331b-4a9b-aed7-e64004bfcb68 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8982024-08-28T02:16:11.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f654c386-331b-4a9b-aed7-e64004bfcb68 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
8992024-08-28T02:16:11.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2091 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f654c386-331b-4a9b-aed7-e64004bfcb68 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9002024-08-28T02:16:12.524ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 08c78bda-e71d-40d3-9b80-7a7dfc772ae2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9012024-08-28T02:16:12.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 08c78bda-e71d-40d3-9b80-7a7dfc772ae2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9022024-08-28T02:16:12.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2283 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 08c78bda-e71d-40d3-9b80-7a7dfc772ae2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9032024-08-28T02:16:13.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8e153d49-e8a6-42ab-8285-c4ae5697fb7e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9042024-08-28T02:16:13.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8e153d49-e8a6-42ab-8285-c4ae5697fb7e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9052024-08-28T02:16:13.550ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3040 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8e153d49-e8a6-42ab-8285-c4ae5697fb7e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9062024-08-28T02:16:14.569ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 09e255d8-f4cf-4333-a71b-83d21025e59b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9072024-08-28T02:16:14.570ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 09e255d8-f4cf-4333-a71b-83d21025e59b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9082024-08-28T02:16:14.572ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2151 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 09e255d8-f4cf-4333-a71b-83d21025e59b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9092024-08-28T02:16:15.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c0c89648-a0b2-4735-b916-36274ea731f6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9102024-08-28T02:16:15.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c0c89648-a0b2-4735-b916-36274ea731f6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9112024-08-28T02:16:15.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2487 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c0c89648-a0b2-4735-b916-36274ea731f6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9122024-08-28T02:16:15.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1508 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = ed5ef2fa-5e9e-46f6-aac0-5a70b831633e response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
9132024-08-28T02:16:16.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ddafb4d7-0d8f-47c7-b3f5-783a4d3b13a3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9142024-08-28T02:16:16.617ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ddafb4d7-0d8f-47c7-b3f5-783a4d3b13a3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9152024-08-28T02:16:16.619ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1840 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ddafb4d7-0d8f-47c7-b3f5-783a4d3b13a3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
9162024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
9172024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
9182024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
9192024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9202024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9212024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
9222024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
9232024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9242024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
9252024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9262024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
9272024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
9282024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9292024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9302024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
9312024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
9322024-08-28T02:16:17.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
9332024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9342024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
9352024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
9362024-08-28T02:16:17.145ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758
9372024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.7\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758"}
9382024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
9392024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
9402024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9412024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
9422024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
9432024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9442024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9452024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9462024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
9472024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
9482024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
9492024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
9502024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9512024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9522024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9532024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
9542024-08-28T02:16:17.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
9552024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9562024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
9572024-08-28T02:16:17.145ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
9582024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9592024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
9602024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9612024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
9622024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9632024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
9642024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9652024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
9662024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
9672024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9682024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9692024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
9702024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
9712024-08-28T02:16:17.145ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-28 02:06:17.145544584 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9722024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9732024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
9742024-08-28T02:16:17.145ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
9752024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9762024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
9772024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9782024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9792024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9802024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9812024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9822024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9832024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9842024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9852024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9862024-08-28T02:16:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
9872024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
9882024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9892024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9902024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9912024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9922024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9932024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9942024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9952024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9962024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9972024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
9982024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9992024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10002024-08-28T02:16:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("01868911-e8e4-4a77-b105-b83672ce13b2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c854ff-da18-4c2a-9d2f-7e15248c76a8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1185, tv_nsec: 304438549 } })
10012024-08-28T02:16:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6926ac43-4e2e-4378-9bce-42019ec72a30.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1185, tv_nsec: 304448899 } })
10022024-08-28T02:16:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10032024-08-28T02:16:17.147ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10042024-08-28T02:16:17.147ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10052024-08-28T02:16:17.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.287796ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
10062024-08-28T02:16:17.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10072024-08-28T02:16:17.148ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
10082024-08-28T02:16:17.148ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.664343ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
10092024-08-28T02:16:17.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10102024-08-28T02:16:17.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10112024-08-28T02:16:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.269235ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10122024-08-28T02:16:17.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10132024-08-28T02:16:17.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10142024-08-28T02:16:17.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10152024-08-28T02:16:17.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10162024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10172024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10182024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10192024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10202024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10212024-08-28T02:16:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10222024-08-28T02:16:17.152ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1185, tv_nsec: 310899117 } })
10232024-08-28T02:16:17.153ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
10242024-08-28T02:16:17.154ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1185, tv_nsec: 312675372 } })
10252024-08-28T02:16:17.154ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 8.994459ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
10262024-08-28T02:16:17.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10272024-08-28T02:16:17.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10282024-08-28T02:16:17.155ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
10292024-08-28T02:16:17.156ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10302024-08-28T02:16:17.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10312024-08-28T02:16:17.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10322024-08-28T02:16:17.158ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332024-08-28T02:16:17.158ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10342024-08-28T02:16:17.160ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10352024-08-28T02:16:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10362024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
10372024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10382024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10392024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10402024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10412024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10422024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10432024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10442024-08-28T02:16:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10452024-08-28T02:16:17.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10462024-08-28T02:16:17.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10472024-08-28T02:16:17.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10482024-08-28T02:16:17.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10492024-08-28T02:16:17.164ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10502024-08-28T02:16:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10512024-08-28T02:16:17.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10522024-08-28T02:16:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10532024-08-28T02:16:17.167ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 21.921075ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10542024-08-28T02:16:17.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10552024-08-28T02:16:17.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10562024-08-28T02:16:17.168ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10572024-08-28T02:16:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10582024-08-28T02:16:17.169ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1185, tv_nsec: 328223611 } })
10592024-08-28T02:16:17.170ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-28 02:06:17.145544584 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10602024-08-28T02:16:17.170ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 25.134729ms iteration = 6 status = {"expiration":"2024-08-28T02:06:17.145544584Z","pruned":[]}
10612024-08-28T02:16:17.171ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
10622024-08-28T02:16:17.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10632024-08-28T02:16:17.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10642024-08-28T02:16:17.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10652024-08-28T02:16:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10662024-08-28T02:16:17.173ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10672024-08-28T02:16:17.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10682024-08-28T02:16:17.175ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10692024-08-28T02:16:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10702024-08-28T02:16:17.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10712024-08-28T02:16:17.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10722024-08-28T02:16:17.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10732024-08-28T02:16:17.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10742024-08-28T02:16:17.177ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10752024-08-28T02:16:17.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10762024-08-28T02:16:17.178ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10772024-08-28T02:16:17.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10782024-08-28T02:16:17.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10792024-08-28T02:16:17.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
10802024-08-28T02:16:17.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10812024-08-28T02:16:17.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10822024-08-28T02:16:17.182ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10832024-08-28T02:16:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10842024-08-28T02:16:17.183ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10852024-08-28T02:16:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
10862024-08-28T02:16:17.188ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10872024-08-28T02:16:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10882024-08-28T02:16:17.190ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10892024-08-28T02:16:17.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10902024-08-28T02:16:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10912024-08-28T02:16:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10922024-08-28T02:16:17.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10932024-08-28T02:16:17.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10942024-08-28T02:16:17.191ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10952024-08-28T02:16:17.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10962024-08-28T02:16:17.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10972024-08-28T02:16:17.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10982024-08-28T02:16:17.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10992024-08-28T02:16:17.194ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
11002024-08-28T02:16:17.194ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.163896ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
11012024-08-28T02:16:17.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11022024-08-28T02:16:17.194ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11032024-08-28T02:16:17.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11042024-08-28T02:16:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11052024-08-28T02:16:17.200ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11062024-08-28T02:16:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11072024-08-28T02:16:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11082024-08-28T02:16:17.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11092024-08-28T02:16:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11102024-08-28T02:16:17.203ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.739517ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
11112024-08-28T02:16:17.203ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
11122024-08-28T02:16:17.203ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11132024-08-28T02:16:17.203ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11142024-08-28T02:16:17.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11152024-08-28T02:16:17.204ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11162024-08-28T02:16:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11172024-08-28T02:16:17.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11182024-08-28T02:16:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11192024-08-28T02:16:17.206ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
11202024-08-28T02:16:17.206ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
11212024-08-28T02:16:17.206ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 61.166154ms iteration = 6 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
11222024-08-28T02:16:17.206ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11232024-08-28T02:16:17.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11242024-08-28T02:16:17.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11252024-08-28T02:16:17.207ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
11262024-08-28T02:16:17.207ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
11272024-08-28T02:16:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 62.247956ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
11282024-08-28T02:16:17.208ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11292024-08-28T02:16:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11302024-08-28T02:16:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11312024-08-28T02:16:17.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11322024-08-28T02:16:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11332024-08-28T02:16:17.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11342024-08-28T02:16:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11352024-08-28T02:16:17.211ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11362024-08-28T02:16:17.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11372024-08-28T02:16:17.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11382024-08-28T02:16:17.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11392024-08-28T02:16:17.214ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11402024-08-28T02:16:17.214ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11412024-08-28T02:16:17.214ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11422024-08-28T02:16:17.214ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11432024-08-28T02:16:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11442024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
11452024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
11462024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
11472024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11482024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11492024-08-28T02:16:17.216ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11502024-08-28T02:16:17.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11512024-08-28T02:16:17.217ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11522024-08-28T02:16:17.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11532024-08-28T02:16:17.218ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11542024-08-28T02:16:17.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11552024-08-28T02:16:17.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11562024-08-28T02:16:17.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11572024-08-28T02:16:17.220ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11582024-08-28T02:16:17.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
11592024-08-28T02:16:17.220ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11602024-08-28T02:16:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11612024-08-28T02:16:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11622024-08-28T02:16:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11632024-08-28T02:16:17.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11642024-08-28T02:16:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11652024-08-28T02:16:17.222ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662024-08-28T02:16:17.224ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 78.868016ms iteration = 11 status = 0
11672024-08-28T02:16:17.224ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11682024-08-28T02:16:17.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11692024-08-28T02:16:17.225ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702024-08-28T02:16:17.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11712024-08-28T02:16:17.226ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11722024-08-28T02:16:17.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11732024-08-28T02:16:17.227ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11742024-08-28T02:16:17.227ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11752024-08-28T02:16:17.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11762024-08-28T02:16:17.229ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11772024-08-28T02:16:17.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11782024-08-28T02:16:17.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11792024-08-28T02:16:17.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11802024-08-28T02:16:17.234ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11812024-08-28T02:16:17.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11822024-08-28T02:16:17.235ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 90.050999ms iteration = 11 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
11832024-08-28T02:16:17.236ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11842024-08-28T02:16:17.236ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 91.508937ms iteration = 11 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
11852024-08-28T02:16:17.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11862024-08-28T02:16:17.237ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
11872024-08-28T02:16:17.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.286163ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
11882024-08-28T02:16:17.237ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11892024-08-28T02:16:17.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11902024-08-28T02:16:17.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11912024-08-28T02:16:17.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11922024-08-28T02:16:17.240ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11932024-08-28T02:16:17.242ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
11942024-08-28T02:16:17.242ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11952024-08-28T02:16:17.242ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 96.776662ms iteration = 11 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
11962024-08-28T02:16:17.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
11972024-08-28T02:16:17.244ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11982024-08-28T02:16:17.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11992024-08-28T02:16:17.247ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12002024-08-28T02:16:17.247ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12012024-08-28T02:16:17.247ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.25397ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12022024-08-28T02:16:17.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12032024-08-28T02:16:17.250ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12042024-08-28T02:16:17.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12052024-08-28T02:16:17.251ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12062024-08-28T02:16:17.251ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12072024-08-28T02:16:17.251ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12082024-08-28T02:16:17.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12092024-08-28T02:16:17.258ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12102024-08-28T02:16:17.258ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 0 version = 2
12112024-08-28T02:16:17.258ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12122024-08-28T02:16:17.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12132024-08-28T02:16:17.264ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-28T02:11:15.536604Z, time_modified: 2024-08-28T02:11:15.536604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }) }
12142024-08-28T02:16:17.264ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 118.86145ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]}},"default_endpoint":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
12152024-08-28T02:16:17.275ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12162024-08-28T02:16:17.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12172024-08-28T02:16:17.278ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
12182024-08-28T02:16:17.295ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
12192024-08-28T02:16:17.305ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 160.264432ms iteration = 14 status = {"physical_disks_added":0}
12202024-08-28T02:16:17.352ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
12212024-08-28T02:16:17.352ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12222024-08-28T02:16:17.353ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0f0cb80-9192-44ad-a3f7-bc20a2b872aa", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
12232024-08-28T02:16:17.353ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12242024-08-28T02:16:17.355ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
12252024-08-28T02:16:17.413ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12262024-08-28T02:16:17.414ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6ac8f435-d994-4ce8-a68d-800d72823e77", "content-length": "372", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
12272024-08-28T02:16:17.414ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12282024-08-28T02:16:17.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12292024-08-28T02:16:17.416ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12302024-08-28T02:16:17.416ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12312024-08-28T02:16:17.417ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12322024-08-28T02:16:17.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12332024-08-28T02:16:17.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12342024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12352024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12362024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12372024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12382024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12392024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12402024-08-28T02:16:17.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12412024-08-28T02:16:17.422ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 1 version = 2
12422024-08-28T02:16:17.422ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12432024-08-28T02:16:17.422ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-28 02:11:35.148647 UTC dns_group = external generation = 2
12442024-08-28T02:16:17.422ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 277.045381ms iteration = 7 status = {"generation":2}
12452024-08-28T02:16:17.423ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
12462024-08-28T02:16:17.423ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12472024-08-28T02:16:17.423ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-28 02:11:34.295733 UTC dns_group = internal generation = 1
12482024-08-28T02:16:17.423ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 278.289746ms iteration = 7 status = {"generation":1}
12492024-08-28T02:16:17.423ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12502024-08-28T02:16:17.424ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3e20685e-2107-4e3f-9f3f-ec647a402621", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
12512024-08-28T02:16:17.426ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 280.583142ms iteration = 12 status = {}
12522024-08-28T02:16:17.440ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12532024-08-28T02:16:17.443ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
12542024-08-28T02:16:17.443ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12552024-08-28T02:16:17.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12562024-08-28T02:16:17.443ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12572024-08-28T02:16:17.444ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7ea9e45-0bf3-4599-a223-0147acd02b14", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12582024-08-28T02:16:17.444ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12592024-08-28T02:16:17.445ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "31bb9ec5-24c4-4fb1-9dc9-83f51cb3bb0a", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12602024-08-28T02:16:17.445ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12612024-08-28T02:16:17.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = afcc644f-7a30-410d-85d5-4040c610238b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
12622024-08-28T02:16:17.446ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = afcc644f-7a30-410d-85d5-4040c610238b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
12632024-08-28T02:16:17.447ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12642024-08-28T02:16:17.447ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12652024-08-28T02:16:17.447ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 302.732101ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
12662024-08-28T02:16:17.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12672024-08-28T02:16:17.448ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2621 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = afcc644f-7a30-410d-85d5-4040c610238b response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
12682024-08-28T02:16:17.449ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12692024-08-28T02:16:17.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12702024-08-28T02:16:17.451ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12712024-08-28T02:16:17.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12722024-08-28T02:16:17.453ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12732024-08-28T02:16:17.453ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
12742024-08-28T02:16:17.453ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12752024-08-28T02:16:17.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12762024-08-28T02:16:17.454ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f947b690-8015-49f7-b6e6-aa9be318404c", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
12772024-08-28T02:16:17.454ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12782024-08-28T02:16:17.454ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
12792024-08-28T02:16:17.454ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12802024-08-28T02:16:17.454ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ac89c76-11d0-4762-b94c-41cd36f3befe", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
12812024-08-28T02:16:17.454ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12822024-08-28T02:16:17.456ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12832024-08-28T02:16:17.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12842024-08-28T02:16:17.456ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12852024-08-28T02:16:17.456ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12862024-08-28T02:16:17.464ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12872024-08-28T02:16:17.482ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12882024-08-28T02:16:17.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12892024-08-28T02:16:17.497ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12902024-08-28T02:16:17.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12912024-08-28T02:16:17.510ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12922024-08-28T02:16:17.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12932024-08-28T02:16:17.511ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12942024-08-28T02:16:17.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
12952024-08-28T02:16:17.515ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12962024-08-28T02:16:17.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12972024-08-28T02:16:17.516ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12982024-08-28T02:16:17.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12992024-08-28T02:16:17.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002024-08-28T02:16:17.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13012024-08-28T02:16:17.525ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13022024-08-28T02:16:17.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13032024-08-28T02:16:17.541ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13042024-08-28T02:16:17.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13052024-08-28T02:16:17.543ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062024-08-28T02:16:17.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13072024-08-28T02:16:17.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13082024-08-28T02:16:17.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13092024-08-28T02:16:17.550ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13102024-08-28T02:16:17.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13112024-08-28T02:16:17.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13122024-08-28T02:16:17.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13132024-08-28T02:16:17.552ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
13142024-08-28T02:16:17.552ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13152024-08-28T02:16:17.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13162024-08-28T02:16:17.554ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13172024-08-28T02:16:17.555ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
13182024-08-28T02:16:17.555ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13192024-08-28T02:16:17.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13202024-08-28T02:16:17.557ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
13212024-08-28T02:16:17.557ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
13222024-08-28T02:16:17.557ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
13232024-08-28T02:16:17.569ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13242024-08-28T02:16:17.570ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "82ec959d-8b81-47c6-b344-22446471ce28", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13252024-08-28T02:16:17.570ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13262024-08-28T02:16:17.572ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b25b5998-86cb-4acd-b903-55187e078d71", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13272024-08-28T02:16:17.572ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13282024-08-28T02:16:17.573ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 428.007459ms iteration = 11 status = {}
13292024-08-28T02:16:17.573ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13302024-08-28T02:16:17.573ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d9e87029-3e8d-436a-80db-07822ef0ff99", "content-length": "1", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
13312024-08-28T02:16:17.574ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "baa4ef24-593e-4fab-be7d-5eee684e39ac", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13322024-08-28T02:16:17.574ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13332024-08-28T02:16:17.575ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13342024-08-28T02:16:17.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13352024-08-28T02:16:17.577ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13362024-08-28T02:16:17.577ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 431.786054ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13372024-08-28T02:16:17.578ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 432.669222ms iteration = 11 status = null
13382024-08-28T02:16:17.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 32320797-50cf-40d7-8b0b-85bae9d067b1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13392024-08-28T02:16:17.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 32320797-50cf-40d7-8b0b-85bae9d067b1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13402024-08-28T02:16:17.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2502 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 32320797-50cf-40d7-8b0b-85bae9d067b1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13412024-08-28T02:16:17.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13422024-08-28T02:16:17.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13432024-08-28T02:16:17.660ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
13442024-08-28T02:16:17.660ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13452024-08-28T02:16:17.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13462024-08-28T02:16:17.670ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13472024-08-28T02:16:17.670ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "edf1e6bb-e7b9-430c-8e3c-e495cccb1c88", "content-length": "121", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
13482024-08-28T02:16:17.670ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13492024-08-28T02:16:17.670ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
13502024-08-28T02:16:17.671ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13512024-08-28T02:16:17.671ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13522024-08-28T02:16:17.671ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
13532024-08-28T02:16:17.671ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
13542024-08-28T02:16:17.671ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
13552024-08-28T02:16:17.671ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13562024-08-28T02:16:17.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13572024-08-28T02:16:17.673ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13582024-08-28T02:16:17.673ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13592024-08-28T02:16:17.674ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ab2434cf-0f3d-4971-887f-e2f212c73dd1", "content-length": "2", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
13602024-08-28T02:16:17.674ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13612024-08-28T02:16:17.674ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "514a1185-86be-467a-9992-21a7d87a9e30", "content-length": "691", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
13622024-08-28T02:16:17.674ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
13632024-08-28T02:16:17.674ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
13642024-08-28T02:16:17.674ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
13652024-08-28T02:16:17.674ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b318fc73-f818-4101-bd61-91963e13c0c8", "content-length": "51", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
13662024-08-28T02:16:17.674ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
13672024-08-28T02:16:17.674ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13682024-08-28T02:16:17.675ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 529.737339ms iteration = 11 status = {}
13692024-08-28T02:16:17.712ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 40b49400-5cd0-4d5c-a006-b241a1ca8261, time_created: 2024-08-28T02:11:30.919762Z, time_modified: 2024-08-28T02:11:30.919762Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
13702024-08-28T02:16:17.712ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
13712024-08-28T02:16:17.712ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13722024-08-28T02:16:17.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13732024-08-28T02:16:17.714ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13742024-08-28T02:16:17.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13752024-08-28T02:16:17.742ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762024-08-28T02:16:17.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
13772024-08-28T02:16:17.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6d161c63-582a-4bcf-82de-94140e9bdd1f", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
13782024-08-28T02:16:17.766ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
13792024-08-28T02:16:17.766ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13802024-08-28T02:16:17.767ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a88073e8-980a-4071-8d92-e9ad3e76ef68", "content-length": "667", "date": "Wed, 28 Aug 2024 02:16:16 GMT"} })
13812024-08-28T02:16:17.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13822024-08-28T02:16:17.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13832024-08-28T02:16:17.770ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13842024-08-28T02:16:17.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13852024-08-28T02:16:17.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13862024-08-28T02:16:17.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13872024-08-28T02:16:17.776ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13882024-08-28T02:16:17.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13892024-08-28T02:16:17.776ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13902024-08-28T02:16:17.777ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "854c0ce3-87dd-4f2c-b49b-724837dd3db3", "date": "Wed, 28 Aug 2024 02:16:17 GMT"} })
13912024-08-28T02:16:17.778ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
13922024-08-28T02:16:17.778ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 633.107091ms
13932024-08-28T02:16:17.778ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 633.115812ms iteration = 2 status = {}
13942024-08-28T02:16:17.784ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 638.834465ms iteration = 11 status = {}
13952024-08-28T02:16:18.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d76e41b-d7b7-4e4f-939d-6e37a0f5ea95 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13962024-08-28T02:16:18.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d76e41b-d7b7-4e4f-939d-6e37a0f5ea95 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13972024-08-28T02:16:18.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2380 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d76e41b-d7b7-4e4f-939d-6e37a0f5ea95 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13982024-08-28T02:16:19.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f0b1045d-5130-4259-9d4f-0d1168ba243b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
13992024-08-28T02:16:19.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f0b1045d-5130-4259-9d4f-0d1168ba243b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14002024-08-28T02:16:19.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2208 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f0b1045d-5130-4259-9d4f-0d1168ba243b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14012024-08-28T02:16:20.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 93cf1749-5b72-47e8-aa18-0d91b3c57eb6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14022024-08-28T02:16:20.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 93cf1749-5b72-47e8-aa18-0d91b3c57eb6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14032024-08-28T02:16:20.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2633 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 93cf1749-5b72-47e8-aa18-0d91b3c57eb6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14042024-08-28T02:16:21.203ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:48381
14052024-08-28T02:16:21.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48381 req_id = 56b04cde-97bc-4cb4-b1a8-f0946b2ef872 roles = RoleSet { roles: {} } uri = /metrics/producers
14062024-08-28T02:16:21.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48381 req_id = 56b04cde-97bc-4cb4-b1a8-f0946b2ef872 resource = Database result = Ok(()) uri = /metrics/producers
14072024-08-28T02:16:21.237ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
14082024-08-28T02:16:21.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48381 req_id = 56b04cde-97bc-4cb4-b1a8-f0946b2ef872 roles = RoleSet { roles: {} } uri = /metrics/producers
14092024-08-28T02:16:21.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48381 req_id = 56b04cde-97bc-4cb4-b1a8-f0946b2ef872 resource = Database result = Ok(()) uri = /metrics/producers
14102024-08-28T02:16:21.267ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
14112024-08-28T02:16:21.267ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d6871eba-4d5c-430b-9dac-ad393aeb3feb", "date": "Wed, 28 Aug 2024 02:16:21 GMT"} })
14122024-08-28T02:16:21.267ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = d65da7b7-eebe-4dd3-87b9-b0fa344c777a
14132024-08-28T02:16:21.268ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 64791 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:48381 req_id = 56b04cde-97bc-4cb4-b1a8-f0946b2ef872 response_code = 201 uri = /metrics/producers
14142024-08-28T02:16:21.728ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4f46a5b-a97e-4eb8-9f25-8ad81483a7e3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14152024-08-28T02:16:21.729ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4f46a5b-a97e-4eb8-9f25-8ad81483a7e3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14162024-08-28T02:16:21.730ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2541 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4f46a5b-a97e-4eb8-9f25-8ad81483a7e3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14172024-08-28T02:16:22.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 851a8ec0-a0a5-40b9-b9d0-b12e80c8f8bb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14182024-08-28T02:16:22.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 851a8ec0-a0a5-40b9-b9d0-b12e80c8f8bb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14192024-08-28T02:16:22.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2299 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 851a8ec0-a0a5-40b9-b9d0-b12e80c8f8bb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14202024-08-28T02:16:23.444ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:53099
14212024-08-28T02:16:23.444ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53099 req_id = aac21907-fc38-4c9b-8d1c-388d62d4e29d roles = RoleSet { roles: {} } uri = /metrics/producers
14222024-08-28T02:16:23.444ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53099 req_id = aac21907-fc38-4c9b-8d1c-388d62d4e29d resource = Database result = Ok(()) uri = /metrics/producers
14232024-08-28T02:16:23.477ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
14242024-08-28T02:16:23.477ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53099 req_id = aac21907-fc38-4c9b-8d1c-388d62d4e29d roles = RoleSet { roles: {} } uri = /metrics/producers
14252024-08-28T02:16:23.478ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53099 req_id = aac21907-fc38-4c9b-8d1c-388d62d4e29d resource = Database result = Ok(()) uri = /metrics/producers
14262024-08-28T02:16:23.507ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
14272024-08-28T02:16:23.508ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e735ba59-7b9a-4042-b96e-9b710bc59018", "date": "Wed, 28 Aug 2024 02:16:23 GMT"} })
14282024-08-28T02:16:23.508ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
14292024-08-28T02:16:23.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65072 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:53099 req_id = aac21907-fc38-4c9b-8d1c-388d62d4e29d response_code = 201 uri = /metrics/producers
14302024-08-28T02:16:23.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a9a8779-b3b0-40a6-bc10-65bb92b06612 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14312024-08-28T02:16:23.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a9a8779-b3b0-40a6-bc10-65bb92b06612 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14322024-08-28T02:16:23.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2409 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a9a8779-b3b0-40a6-bc10-65bb92b06612 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14332024-08-28T02:16:24.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1e98c3e-ff03-45c4-80ba-1020b749125c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14342024-08-28T02:16:24.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1e98c3e-ff03-45c4-80ba-1020b749125c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14352024-08-28T02:16:24.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2122 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1e98c3e-ff03-45c4-80ba-1020b749125c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14362024-08-28T02:16:25.814ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 66c5992c-e473-42c9-b25a-0d43a8c7c16d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14372024-08-28T02:16:25.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 66c5992c-e473-42c9-b25a-0d43a8c7c16d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14382024-08-28T02:16:25.816ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1903 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 66c5992c-e473-42c9-b25a-0d43a8c7c16d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14392024-08-28T02:16:25.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1585 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 466e3bcc-0910-41c5-bfde-d1701196fc1d response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
14402024-08-28T02:16:26.838ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7ab7944-a93d-4c3a-987d-b4ea2aee5af4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14412024-08-28T02:16:26.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7ab7944-a93d-4c3a-987d-b4ea2aee5af4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14422024-08-28T02:16:26.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2765 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7ab7944-a93d-4c3a-987d-b4ea2aee5af4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14432024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
14442024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14452024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
14462024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14472024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14482024-08-28T02:16:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14492024-08-28T02:16:27.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14502024-08-28T02:16:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14512024-08-28T02:16:27.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14522024-08-28T02:16:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14532024-08-28T02:16:27.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14542024-08-28T02:16:27.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14552024-08-28T02:16:27.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14562024-08-28T02:16:27.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
14572024-08-28T02:16:27.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14582024-08-28T02:16:27.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14592024-08-28T02:16:27.171ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
14602024-08-28T02:16:27.171ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.67672ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
14612024-08-28T02:16:27.172ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14622024-08-28T02:16:27.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14632024-08-28T02:16:27.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.047264ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14642024-08-28T02:16:27.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e56c339f-fb3c-4e65-b895-46c6600f2864 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14652024-08-28T02:16:27.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e56c339f-fb3c-4e65-b895-46c6600f2864 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14662024-08-28T02:16:27.859ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1830 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e56c339f-fb3c-4e65-b895-46c6600f2864 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14672024-08-28T02:16:28.879ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c541d477-5298-44d6-a05e-cd5a85e8e390 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14682024-08-28T02:16:28.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c541d477-5298-44d6-a05e-cd5a85e8e390 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14692024-08-28T02:16:28.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2063 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c541d477-5298-44d6-a05e-cd5a85e8e390 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14702024-08-28T02:16:29.901ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 44aa8c03-1d1a-4fe4-9d75-19235e3de855 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14712024-08-28T02:16:29.902ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 44aa8c03-1d1a-4fe4-9d75-19235e3de855 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14722024-08-28T02:16:29.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2353 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 44aa8c03-1d1a-4fe4-9d75-19235e3de855 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14732024-08-28T02:16:30.922ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b9f5646-5589-4de2-a599-924f388508e0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14742024-08-28T02:16:30.923ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b9f5646-5589-4de2-a599-924f388508e0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14752024-08-28T02:16:30.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1920 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b9f5646-5589-4de2-a599-924f388508e0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14762024-08-28T02:16:31.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e59914ac-7857-4886-9d47-cd766a52dc2e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14772024-08-28T02:16:31.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e59914ac-7857-4886-9d47-cd766a52dc2e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14782024-08-28T02:16:31.945ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1968 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e59914ac-7857-4886-9d47-cd766a52dc2e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14792024-08-28T02:16:32.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45fd612f-4022-45e6-8d42-3dd6a9d7559d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14802024-08-28T02:16:32.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45fd612f-4022-45e6-8d42-3dd6a9d7559d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14812024-08-28T02:16:32.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2424 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 45fd612f-4022-45e6-8d42-3dd6a9d7559d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14822024-08-28T02:16:33.565ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:34140
14832024-08-28T02:16:33.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34140 req_id = 5786ab41-365b-4fd3-85b4-86396a6a426a roles = RoleSet { roles: {} } uri = /metrics/producers
14842024-08-28T02:16:33.565ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34140 req_id = 5786ab41-365b-4fd3-85b4-86396a6a426a resource = Database result = Ok(()) uri = /metrics/producers
14852024-08-28T02:16:33.598ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
14862024-08-28T02:16:33.598ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34140 req_id = 5786ab41-365b-4fd3-85b4-86396a6a426a roles = RoleSet { roles: {} } uri = /metrics/producers
14872024-08-28T02:16:33.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34140 req_id = 5786ab41-365b-4fd3-85b4-86396a6a426a resource = Database result = Ok(()) uri = /metrics/producers
14882024-08-28T02:16:33.629ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
14892024-08-28T02:16:33.630ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "45b26a97-cb44-43dd-96f3-d7eaa775deaf", "date": "Wed, 28 Aug 2024 02:16:33 GMT"} })
14902024-08-28T02:16:33.630ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = f5d97867-5abf-481d-a256-03ebe539fcd7
14912024-08-28T02:16:33.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65661 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:34140 req_id = 5786ab41-365b-4fd3-85b4-86396a6a426a response_code = 201 uri = /metrics/producers
14922024-08-28T02:16:33.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc4afc38-8fb8-486d-a1b7-eb9bb384c0b5 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14932024-08-28T02:16:33.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc4afc38-8fb8-486d-a1b7-eb9bb384c0b5 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14942024-08-28T02:16:33.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2077 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc4afc38-8fb8-486d-a1b7-eb9bb384c0b5 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14952024-08-28T02:16:35.007ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 49ab3bc6-33d8-44fb-9e50-b96f10f81786 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14962024-08-28T02:16:35.007ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 49ab3bc6-33d8-44fb-9e50-b96f10f81786 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14972024-08-28T02:16:35.009ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 49ab3bc6-33d8-44fb-9e50-b96f10f81786 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
14982024-08-28T02:16:35.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2087 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = f0b12194-3092-4696-bcdf-2059e117443c response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
14992024-08-28T02:16:35.916ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
15002024-08-28T02:16:35.916ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
15012024-08-28T02:16:35.946ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
15022024-08-28T02:16:35.946ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:48163
15032024-08-28T02:16:35.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:48163 req_id = 7d90a14d-4bac-474b-9ab0-d19d5b7d899b roles = RoleSet { roles: {} } uri = /metrics/producers
15042024-08-28T02:16:35.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:48163 req_id = 7d90a14d-4bac-474b-9ab0-d19d5b7d899b resource = Database result = Ok(()) uri = /metrics/producers
15052024-08-28T02:16:35.980ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
15062024-08-28T02:16:35.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:48163 req_id = 7d90a14d-4bac-474b-9ab0-d19d5b7d899b roles = RoleSet { roles: {} } uri = /metrics/producers
15072024-08-28T02:16:35.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:48163 req_id = 7d90a14d-4bac-474b-9ab0-d19d5b7d899b resource = Database result = Ok(()) uri = /metrics/producers
15082024-08-28T02:16:35.985ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
15092024-08-28T02:16:35.986ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e92970fc-4bf5-4a6e-af01-2d2c60ce53aa", "date": "Wed, 28 Aug 2024 02:16:35 GMT"} })
15102024-08-28T02:16:35.986ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
15112024-08-28T02:16:35.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40299 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:48163 req_id = 7d90a14d-4bac-474b-9ab0-d19d5b7d899b response_code = 201 uri = /metrics/producers
15122024-08-28T02:16:35.987ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "7d90a14d-4bac-474b-9ab0-d19d5b7d899b", "content-length": "41", "date": "Wed, 28 Aug 2024 02:16:35 GMT"} })
15132024-08-28T02:16:35.989ZDEBGnexus (producer-registration-task): registered with nexus successfully
15142024-08-28T02:16:35.989ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
15152024-08-28T02:16:36.028ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 36f4294d-c764-4a19-a135-7a3119de3411 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15162024-08-28T02:16:36.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 36f4294d-c764-4a19-a135-7a3119de3411 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15172024-08-28T02:16:36.031ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2187 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 36f4294d-c764-4a19-a135-7a3119de3411 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15182024-08-28T02:16:37.050ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c75e8ed0-f874-435d-bf9b-69d713ffd181 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15192024-08-28T02:16:37.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c75e8ed0-f874-435d-bf9b-69d713ffd181 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15202024-08-28T02:16:37.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2504 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c75e8ed0-f874-435d-bf9b-69d713ffd181 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15212024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
15222024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
15232024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15242024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15252024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15262024-08-28T02:16:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15272024-08-28T02:16:37.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15282024-08-28T02:16:37.148ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15292024-08-28T02:16:37.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15302024-08-28T02:16:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15312024-08-28T02:16:37.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15322024-08-28T02:16:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15332024-08-28T02:16:37.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15342024-08-28T02:16:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15352024-08-28T02:16:37.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15362024-08-28T02:16:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
15372024-08-28T02:16:37.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15382024-08-28T02:16:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15392024-08-28T02:16:37.163ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
15402024-08-28T02:16:37.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.772153ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
15412024-08-28T02:16:37.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.444294ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15422024-08-28T02:16:38.072ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 55fa5fff-f2cd-4bdd-a991-97320f57a7ee roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15432024-08-28T02:16:38.072ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 55fa5fff-f2cd-4bdd-a991-97320f57a7ee resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15442024-08-28T02:16:38.074ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2193 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 55fa5fff-f2cd-4bdd-a991-97320f57a7ee response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15452024-08-28T02:16:39.093ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea81a53e-6981-4cdd-9e44-d8c9b6746c6d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15462024-08-28T02:16:39.093ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea81a53e-6981-4cdd-9e44-d8c9b6746c6d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15472024-08-28T02:16:39.095ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2613 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea81a53e-6981-4cdd-9e44-d8c9b6746c6d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15482024-08-28T02:16:40.115ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4cc96fc-99a8-468d-b868-18b19f5afa9b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15492024-08-28T02:16:40.115ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4cc96fc-99a8-468d-b868-18b19f5afa9b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15502024-08-28T02:16:40.117ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d4cc96fc-99a8-468d-b868-18b19f5afa9b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15512024-08-28T02:16:41.135ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 789bbe0c-fe9b-41b8-bb11-1813150eaf84 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15522024-08-28T02:16:41.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 789bbe0c-fe9b-41b8-bb11-1813150eaf84 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15532024-08-28T02:16:41.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1806 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 789bbe0c-fe9b-41b8-bb11-1813150eaf84 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15542024-08-28T02:16:41.331ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:49384
15552024-08-28T02:16:41.331ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49384 req_id = 68645c22-eea8-4b20-b0e3-e0c2ce7edb05 roles = RoleSet { roles: {} } uri = /metrics/producers
15562024-08-28T02:16:41.331ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49384 req_id = 68645c22-eea8-4b20-b0e3-e0c2ce7edb05 resource = Database result = Ok(()) uri = /metrics/producers
15572024-08-28T02:16:41.363ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
15582024-08-28T02:16:41.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49384 req_id = 68645c22-eea8-4b20-b0e3-e0c2ce7edb05 roles = RoleSet { roles: {} } uri = /metrics/producers
15592024-08-28T02:16:41.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49384 req_id = 68645c22-eea8-4b20-b0e3-e0c2ce7edb05 resource = Database result = Ok(()) uri = /metrics/producers
15602024-08-28T02:16:41.369ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
15612024-08-28T02:16:41.369ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5efd1b65-b6b4-4d1d-bc28-d1df26297ded", "date": "Wed, 28 Aug 2024 02:16:41 GMT"} })
15622024-08-28T02:16:41.369ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 7ab6cf11-bb3f-4b6e-b7eb-0005add95826
15632024-08-28T02:16:41.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38928 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49384 req_id = 68645c22-eea8-4b20-b0e3-e0c2ce7edb05 response_code = 201 uri = /metrics/producers
15642024-08-28T02:16:41.739ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:56436
15652024-08-28T02:16:41.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56436 req_id = b09186de-185d-471e-bdb1-0498c28772ff roles = RoleSet { roles: {} } uri = /metrics/producers
15662024-08-28T02:16:41.739ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56436 req_id = b09186de-185d-471e-bdb1-0498c28772ff resource = Database result = Ok(()) uri = /metrics/producers
15672024-08-28T02:16:41.772ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
15682024-08-28T02:16:41.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56436 req_id = b09186de-185d-471e-bdb1-0498c28772ff roles = RoleSet { roles: {} } uri = /metrics/producers
15692024-08-28T02:16:41.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56436 req_id = b09186de-185d-471e-bdb1-0498c28772ff resource = Database result = Ok(()) uri = /metrics/producers
15702024-08-28T02:16:41.777ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
15712024-08-28T02:16:41.778ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d16fbf00-f358-4d56-b2f5-61f30664f1ba", "date": "Wed, 28 Aug 2024 02:16:41 GMT"} })
15722024-08-28T02:16:41.778ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 86d235a7-2247-4a0d-a3ed-09ecc4c92890
15732024-08-28T02:16:41.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39533 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56436 req_id = b09186de-185d-471e-bdb1-0498c28772ff response_code = 201 uri = /metrics/producers
15742024-08-28T02:16:42.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 23e05665-3c3b-49c0-bb2c-eccb74bab1f1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15752024-08-28T02:16:42.157ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 23e05665-3c3b-49c0-bb2c-eccb74bab1f1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15762024-08-28T02:16:42.158ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1732 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 23e05665-3c3b-49c0-bb2c-eccb74bab1f1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15772024-08-28T02:16:43.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5a1c5b43-a944-42bc-9ba5-e7f9ec9bc312 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15782024-08-28T02:16:43.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5a1c5b43-a944-42bc-9ba5-e7f9ec9bc312 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15792024-08-28T02:16:43.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5a1c5b43-a944-42bc-9ba5-e7f9ec9bc312 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15802024-08-28T02:16:44.200ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bd5fd2ed-0186-455d-bc60-7d854e14d5ee roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15812024-08-28T02:16:44.200ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bd5fd2ed-0186-455d-bc60-7d854e14d5ee resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15822024-08-28T02:16:44.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2214 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bd5fd2ed-0186-455d-bc60-7d854e14d5ee response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15832024-08-28T02:16:45.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ba79c596-4539-475c-af30-ee10eae2f174 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15842024-08-28T02:16:45.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ba79c596-4539-475c-af30-ee10eae2f174 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15852024-08-28T02:16:45.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1750 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ba79c596-4539-475c-af30-ee10eae2f174 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15862024-08-28T02:16:45.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2072 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 116d8062-2b92-4a21-b334-ea9f26c5584c response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
15872024-08-28T02:16:46.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24a4ad03-0bd4-4506-8039-29554acaa8a7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15882024-08-28T02:16:46.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24a4ad03-0bd4-4506-8039-29554acaa8a7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15892024-08-28T02:16:46.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1655 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24a4ad03-0bd4-4506-8039-29554acaa8a7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
15902024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
15912024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15922024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15932024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
15942024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
15952024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15962024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15972024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
15982024-08-28T02:16:47.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
15992024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
16002024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16012024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
16022024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
16032024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16042024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
16052024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16062024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
16072024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16082024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
16092024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
16102024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
16112024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
16122024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16132024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16142024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
16152024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16162024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
16172024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16182024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16192024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
16202024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16212024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
16222024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16232024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16242024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16252024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16262024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
16272024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16282024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16292024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16302024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16312024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16322024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16332024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16342024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16352024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16362024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16372024-08-28T02:16:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16382024-08-28T02:16:47.147ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16392024-08-28T02:16:47.148ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.429548ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
16402024-08-28T02:16:47.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16412024-08-28T02:16:47.149ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16422024-08-28T02:16:47.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16432024-08-28T02:16:47.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16442024-08-28T02:16:47.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16452024-08-28T02:16:47.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16462024-08-28T02:16:47.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16472024-08-28T02:16:47.158ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16482024-08-28T02:16:47.160ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1215, tv_nsec: 318691102 } })
16492024-08-28T02:16:47.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16502024-08-28T02:16:47.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16512024-08-28T02:16:47.161ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
16522024-08-28T02:16:47.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16532024-08-28T02:16:47.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1215, tv_nsec: 322520988 } })
16542024-08-28T02:16:47.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16552024-08-28T02:16:47.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16562024-08-28T02:16:47.165ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
16572024-08-28T02:16:47.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1215, tv_nsec: 325243432 } })
16582024-08-28T02:16:47.167ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
16592024-08-28T02:16:47.174ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
16602024-08-28T02:16:47.174ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16612024-08-28T02:16:47.174ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16622024-08-28T02:16:47.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16632024-08-28T02:16:47.176ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 31.320282ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
16642024-08-28T02:16:47.179ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16652024-08-28T02:16:47.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16662024-08-28T02:16:47.180ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16672024-08-28T02:16:47.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16682024-08-28T02:16:47.182ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16692024-08-28T02:16:47.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16702024-08-28T02:16:47.184ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16712024-08-28T02:16:47.184ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16722024-08-28T02:16:47.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16732024-08-28T02:16:47.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16742024-08-28T02:16:47.186ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16752024-08-28T02:16:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16762024-08-28T02:16:47.188ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16772024-08-28T02:16:47.188ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16782024-08-28T02:16:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16792024-08-28T02:16:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16802024-08-28T02:16:47.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16812024-08-28T02:16:47.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16822024-08-28T02:16:47.192ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 47.090985ms iteration = 12 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
16832024-08-28T02:16:47.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16842024-08-28T02:16:47.193ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16852024-08-28T02:16:47.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16862024-08-28T02:16:47.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16872024-08-28T02:16:47.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16882024-08-28T02:16:47.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16892024-08-28T02:16:47.195ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16902024-08-28T02:16:47.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16912024-08-28T02:16:47.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922024-08-28T02:16:47.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16932024-08-28T02:16:47.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16942024-08-28T02:16:47.197ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16952024-08-28T02:16:47.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16962024-08-28T02:16:47.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16972024-08-28T02:16:47.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16982024-08-28T02:16:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
16992024-08-28T02:16:47.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17002024-08-28T02:16:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17012024-08-28T02:16:47.206ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
17022024-08-28T02:16:47.206ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
17032024-08-28T02:16:47.206ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.43585ms iteration = 12 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
17042024-08-28T02:16:47.211ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
17052024-08-28T02:16:47.211ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.826667ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
17062024-08-28T02:16:47.216ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17072024-08-28T02:16:47.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17082024-08-28T02:16:47.218ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
17092024-08-28T02:16:47.218ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 73.349507ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
17102024-08-28T02:16:47.225ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.471008ms iteration = 12 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
17112024-08-28T02:16:47.227ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17122024-08-28T02:16:47.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17132024-08-28T02:16:47.229ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 84.852545ms iteration = 15 status = {"physical_disks_added":0}
17142024-08-28T02:16:47.230ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17152024-08-28T02:16:47.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17162024-08-28T02:16:47.230ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 85.746162ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17172024-08-28T02:16:47.231ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17182024-08-28T02:16:47.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
17192024-08-28T02:16:47.234ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17202024-08-28T02:16:47.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17212024-08-28T02:16:47.236ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 91.65178ms iteration = 12 status = 0
17222024-08-28T02:16:47.237ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
17232024-08-28T02:16:47.237ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 92.828573ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
17242024-08-28T02:16:47.252ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
17252024-08-28T02:16:47.252ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17262024-08-28T02:16:47.253ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d8c59123-2aec-4aa0-a63e-6d8d761f253b", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17272024-08-28T02:16:47.253ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17282024-08-28T02:16:47.255ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17292024-08-28T02:16:47.256ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17302024-08-28T02:16:47.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17312024-08-28T02:16:47.258ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
17322024-08-28T02:16:47.258ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17332024-08-28T02:16:47.258ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3c721f2c-eab2-44db-92f5-1d4883d90dac", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17342024-08-28T02:16:47.258ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17352024-08-28T02:16:47.259ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
17362024-08-28T02:16:47.259ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17372024-08-28T02:16:47.259ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "807a805f-f6e0-45f3-ac07-14d6471a62c7", "content-length": "26", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17382024-08-28T02:16:47.259ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17392024-08-28T02:16:47.259ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
17402024-08-28T02:16:47.261ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17412024-08-28T02:16:47.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6dd8149d-5f5b-4173-94f8-c9f17794b151 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
17422024-08-28T02:16:47.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6dd8149d-5f5b-4173-94f8-c9f17794b151 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
17432024-08-28T02:16:47.263ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1388 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6dd8149d-5f5b-4173-94f8-c9f17794b151 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
17442024-08-28T02:16:47.289ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17452024-08-28T02:16:47.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17462024-08-28T02:16:47.291ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17472024-08-28T02:16:47.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17482024-08-28T02:16:47.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17492024-08-28T02:16:47.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17502024-08-28T02:16:47.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 2b2f3ece-d875-4b04-9b7f-c9c4506de2c5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
17512024-08-28T02:16:47.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 2b2f3ece-d875-4b04-9b7f-c9c4506de2c5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
17522024-08-28T02:16:47.561ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
17532024-08-28T02:16:47.561ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "790448da-128d-4854-a0f4-c3388aa1c745", "content-length": "1", "date": "Wed, 28 Aug 2024 02:16:46 GMT"} })
17542024-08-28T02:16:47.563ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17552024-08-28T02:16:47.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17562024-08-28T02:16:47.597ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17572024-08-28T02:16:47.598ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 452.911823ms iteration = 12 status = {}
17582024-08-28T02:16:47.598ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "78ea0a0b-ab22-465d-a05d-26bf68cc223d", "content-length": "372", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17592024-08-28T02:16:47.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17602024-08-28T02:16:47.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17612024-08-28T02:16:47.600ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17622024-08-28T02:16:47.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17632024-08-28T02:16:47.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121165 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 2b2f3ece-d875-4b04-9b7f-c9c4506de2c5 response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
17642024-08-28T02:16:47.601ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 456.112837ms iteration = 12 status = null
17652024-08-28T02:16:47.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17662024-08-28T02:16:47.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17672024-08-28T02:16:47.602ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
17682024-08-28T02:16:47.602ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17692024-08-28T02:16:47.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17702024-08-28T02:16:47.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17712024-08-28T02:16:47.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17722024-08-28T02:16:47.607ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17732024-08-28T02:16:47.607ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a113414a-3401-4224-8123-125c24f2d40f", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17742024-08-28T02:16:47.609ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 464.207678ms iteration = 13 status = {}
17752024-08-28T02:16:47.631ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17762024-08-28T02:16:47.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17772024-08-28T02:16:47.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17782024-08-28T02:16:47.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17792024-08-28T02:16:47.671ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17802024-08-28T02:16:47.671ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f379582c-ff8c-4166-92bb-16c5c8d29929", "content-length": "691", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
17812024-08-28T02:16:47.671ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17822024-08-28T02:16:47.671ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17832024-08-28T02:16:47.671ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17842024-08-28T02:16:47.673ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 528.642768ms iteration = 12 status = {}
17852024-08-28T02:16:47.725ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
17862024-08-28T02:16:47.725ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "03f22980-174a-434b-b2fb-afbbf7ad31a7", "content-length": "121", "date": "Wed, 28 Aug 2024 02:16:46 GMT"} })
17872024-08-28T02:16:47.725ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
17882024-08-28T02:16:47.725ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
17892024-08-28T02:16:47.726ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17902024-08-28T02:16:47.726ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17912024-08-28T02:16:47.726ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
17922024-08-28T02:16:47.726ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
17932024-08-28T02:16:47.726ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
17942024-08-28T02:16:47.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17952024-08-28T02:16:47.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17962024-08-28T02:16:47.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
17972024-08-28T02:16:47.728ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b19afcba-d784-4363-a6b8-240750fcfb14", "content-length": "2", "date": "Wed, 28 Aug 2024 02:16:46 GMT"} })
17982024-08-28T02:16:47.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
17992024-08-28T02:16:47.728ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1805f23a-c291-44df-b7b7-55a98d3b7d2d", "content-length": "51", "date": "Wed, 28 Aug 2024 02:16:46 GMT"} })
18002024-08-28T02:16:47.728ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
18012024-08-28T02:16:47.728ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18022024-08-28T02:16:47.827ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "abab3582-edbd-4d8f-91cd-1051fc0a3b3c", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
18032024-08-28T02:16:47.827ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
18042024-08-28T02:16:47.827ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18052024-08-28T02:16:47.828ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "525ade51-52b5-4315-a07f-12c374db9f85", "content-length": "667", "date": "Wed, 28 Aug 2024 02:16:47 GMT"} })
18062024-08-28T02:16:47.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18072024-08-28T02:16:47.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18082024-08-28T02:16:47.832ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18092024-08-28T02:16:47.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18102024-08-28T02:16:47.835ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18112024-08-28T02:16:47.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18122024-08-28T02:16:47.838ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18132024-08-28T02:16:47.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18142024-08-28T02:16:47.849ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 704.665582ms iteration = 12 status = {}
18152024-08-28T02:16:48.284ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bed0eb-377a-4b8a-8764-310a28d0f852 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18162024-08-28T02:16:48.284ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bed0eb-377a-4b8a-8764-310a28d0f852 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18172024-08-28T02:16:48.286ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bed0eb-377a-4b8a-8764-310a28d0f852 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18182024-08-28T02:16:49.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dea34275-873f-4ea4-b8eb-2ef0baead5e6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18192024-08-28T02:16:49.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dea34275-873f-4ea4-b8eb-2ef0baead5e6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18202024-08-28T02:16:49.307ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1835 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dea34275-873f-4ea4-b8eb-2ef0baead5e6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18212024-08-28T02:16:50.160ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:36408
18222024-08-28T02:16:50.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:36408 req_id = ec280131-4e7f-4076-ae46-5cdb9231bfc0 roles = RoleSet { roles: {} } uri = /metrics/producers
18232024-08-28T02:16:50.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:36408 req_id = ec280131-4e7f-4076-ae46-5cdb9231bfc0 resource = Database result = Ok(()) uri = /metrics/producers
18242024-08-28T02:16:50.193ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
18252024-08-28T02:16:50.193ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:36408 req_id = ec280131-4e7f-4076-ae46-5cdb9231bfc0 roles = RoleSet { roles: {} } uri = /metrics/producers
18262024-08-28T02:16:50.194ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:36408 req_id = ec280131-4e7f-4076-ae46-5cdb9231bfc0 resource = Database result = Ok(()) uri = /metrics/producers
18272024-08-28T02:16:50.199ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
18282024-08-28T02:16:50.200ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "36be113d-ccf7-4019-807f-dc1430e56457", "date": "Wed, 28 Aug 2024 02:16:50 GMT"} })
18292024-08-28T02:16:50.200ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 92e923a4-5690-44dd-9005-892bead54a2c
18302024-08-28T02:16:50.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 41377 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:36408 req_id = ec280131-4e7f-4076-ae46-5cdb9231bfc0 response_code = 201 uri = /metrics/producers
18312024-08-28T02:16:50.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 459ab3fb-a0cd-4302-8014-14b712396fde roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18322024-08-28T02:16:50.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 459ab3fb-a0cd-4302-8014-14b712396fde resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18332024-08-28T02:16:50.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2667 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 459ab3fb-a0cd-4302-8014-14b712396fde response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18342024-08-28T02:16:51.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e95615f-841b-439c-b499-3c366269521c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18352024-08-28T02:16:51.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e95615f-841b-439c-b499-3c366269521c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18362024-08-28T02:16:51.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1960 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6e95615f-841b-439c-b499-3c366269521c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18372024-08-28T02:16:52.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4a765407-0d6e-4e4e-8616-1e245e7ea275 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18382024-08-28T02:16:52.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4a765407-0d6e-4e4e-8616-1e245e7ea275 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18392024-08-28T02:16:52.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1820 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4a765407-0d6e-4e4e-8616-1e245e7ea275 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18402024-08-28T02:16:53.389ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 48da4396-0149-490d-880e-9fe6f685f24d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18412024-08-28T02:16:53.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 48da4396-0149-490d-880e-9fe6f685f24d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18422024-08-28T02:16:53.391ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2169 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 48da4396-0149-490d-880e-9fe6f685f24d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18432024-08-28T02:16:54.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a27594ff-18e0-4e7a-88d6-2b5c01772ef3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18442024-08-28T02:16:54.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a27594ff-18e0-4e7a-88d6-2b5c01772ef3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18452024-08-28T02:16:54.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2007 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a27594ff-18e0-4e7a-88d6-2b5c01772ef3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18462024-08-28T02:16:55.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea790719-25fb-4abb-974d-90e96fd5a3d9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18472024-08-28T02:16:55.433ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea790719-25fb-4abb-974d-90e96fd5a3d9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18482024-08-28T02:16:55.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2669 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea790719-25fb-4abb-974d-90e96fd5a3d9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18492024-08-28T02:16:55.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1607 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 1b8251a9-d708-4e7c-8281-ebff8b298a02 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
18502024-08-28T02:16:56.454ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 213b99bb-6268-47cc-b5d2-e948cd9d7af0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18512024-08-28T02:16:56.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 213b99bb-6268-47cc-b5d2-e948cd9d7af0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18522024-08-28T02:16:56.456ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1932 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 213b99bb-6268-47cc-b5d2-e948cd9d7af0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18532024-08-28T02:16:57.144ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
18542024-08-28T02:16:57.144ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18552024-08-28T02:16:57.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
18562024-08-28T02:16:57.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18572024-08-28T02:16:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18582024-08-28T02:16:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18592024-08-28T02:16:57.146ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18602024-08-28T02:16:57.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18612024-08-28T02:16:57.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18622024-08-28T02:16:57.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18632024-08-28T02:16:57.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18642024-08-28T02:16:57.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18652024-08-28T02:16:57.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18662024-08-28T02:16:57.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18672024-08-28T02:16:57.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18682024-08-28T02:16:57.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
18692024-08-28T02:16:57.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18702024-08-28T02:16:57.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18712024-08-28T02:16:57.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
18722024-08-28T02:16:57.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.188021ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
18732024-08-28T02:16:57.173ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.247071ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18742024-08-28T02:16:57.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185dd131-d8ba-4a29-82f3-3fb0a2ab99fb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18752024-08-28T02:16:57.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185dd131-d8ba-4a29-82f3-3fb0a2ab99fb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18762024-08-28T02:16:57.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1667 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 185dd131-d8ba-4a29-82f3-3fb0a2ab99fb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18772024-08-28T02:16:58.497ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ccf064d8-3c72-46d2-8916-af8c5246b89c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18782024-08-28T02:16:58.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ccf064d8-3c72-46d2-8916-af8c5246b89c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18792024-08-28T02:16:58.499ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2247 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ccf064d8-3c72-46d2-8916-af8c5246b89c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18802024-08-28T02:16:59.518ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9935046b-c795-46ef-ab96-e9e29020c816 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18812024-08-28T02:16:59.518ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9935046b-c795-46ef-ab96-e9e29020c816 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18822024-08-28T02:16:59.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9935046b-c795-46ef-ab96-e9e29020c816 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18832024-08-28T02:17:00.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2223e648-cfa3-46a7-b24f-d94d0ce6f740 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18842024-08-28T02:17:00.539ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2223e648-cfa3-46a7-b24f-d94d0ce6f740 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18852024-08-28T02:17:00.541ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3114 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2223e648-cfa3-46a7-b24f-d94d0ce6f740 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18862024-08-28T02:17:01.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 33736fe2-4ecd-46db-87a1-50c89752cb60 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18872024-08-28T02:17:01.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 33736fe2-4ecd-46db-87a1-50c89752cb60 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18882024-08-28T02:17:01.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2154 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 33736fe2-4ecd-46db-87a1-50c89752cb60 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18892024-08-28T02:17:02.581ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1d6652d5-6881-489a-980a-ff185773f052 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18902024-08-28T02:17:02.582ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1d6652d5-6881-489a-980a-ff185773f052 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18912024-08-28T02:17:02.583ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1d6652d5-6881-489a-980a-ff185773f052 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18922024-08-28T02:17:03.602ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b34587f-5b08-43a8-bc27-5f1075debf78 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18932024-08-28T02:17:03.603ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b34587f-5b08-43a8-bc27-5f1075debf78 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18942024-08-28T02:17:03.604ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1770 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3b34587f-5b08-43a8-bc27-5f1075debf78 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18952024-08-28T02:17:04.625ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c786a62b-56df-45fd-80dd-8d37ed7b95db roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18962024-08-28T02:17:04.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c786a62b-56df-45fd-80dd-8d37ed7b95db resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18972024-08-28T02:17:04.626ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c786a62b-56df-45fd-80dd-8d37ed7b95db response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18982024-08-28T02:17:05.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 588fbb57-d8c5-48dc-91a8-9b3b859ba6f0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
18992024-08-28T02:17:05.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 588fbb57-d8c5-48dc-91a8-9b3b859ba6f0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19002024-08-28T02:17:05.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 588fbb57-d8c5-48dc-91a8-9b3b859ba6f0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19012024-08-28T02:17:05.988ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1665 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 81beb457-29a9-422a-9a23-7c88cd7f845c response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
19022024-08-28T02:17:06.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2e8b6d5b-81b6-47e6-84c6-98a4724a3952 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19032024-08-28T02:17:06.669ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2e8b6d5b-81b6-47e6-84c6-98a4724a3952 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19042024-08-28T02:17:06.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2e8b6d5b-81b6-47e6-84c6-98a4724a3952 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19052024-08-28T02:17:07.144ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
19062024-08-28T02:17:07.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
19072024-08-28T02:17:07.144ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19082024-08-28T02:17:07.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19092024-08-28T02:17:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19102024-08-28T02:17:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19112024-08-28T02:17:07.146ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19122024-08-28T02:17:07.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132024-08-28T02:17:07.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19142024-08-28T02:17:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19152024-08-28T02:17:07.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19162024-08-28T02:17:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19172024-08-28T02:17:07.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19182024-08-28T02:17:07.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19192024-08-28T02:17:07.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19202024-08-28T02:17:07.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
19212024-08-28T02:17:07.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19222024-08-28T02:17:07.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19232024-08-28T02:17:07.160ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
19242024-08-28T02:17:07.160ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.852455ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
19252024-08-28T02:17:07.172ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 27.444876ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19262024-08-28T02:17:07.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ba5f103-2456-4977-9d9c-ef554db80238 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19272024-08-28T02:17:07.688ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ba5f103-2456-4977-9d9c-ef554db80238 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19282024-08-28T02:17:07.690ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1536 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5ba5f103-2456-4977-9d9c-ef554db80238 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19292024-08-28T02:17:08.711ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c25b9139-3824-42e5-ae4c-5494a4dcbad1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19302024-08-28T02:17:08.712ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c25b9139-3824-42e5-ae4c-5494a4dcbad1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19312024-08-28T02:17:08.714ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2995 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c25b9139-3824-42e5-ae4c-5494a4dcbad1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19322024-08-28T02:17:09.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0b82deb0-142d-4af7-9f55-7869e7b6f600 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19332024-08-28T02:17:09.733ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0b82deb0-142d-4af7-9f55-7869e7b6f600 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19342024-08-28T02:17:09.734ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1978 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0b82deb0-142d-4af7-9f55-7869e7b6f600 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19352024-08-28T02:17:10.755ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c2c818cc-3987-4f1f-92fd-a13fe4cb1f95 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19362024-08-28T02:17:10.755ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c2c818cc-3987-4f1f-92fd-a13fe4cb1f95 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19372024-08-28T02:17:10.757ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1846 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c2c818cc-3987-4f1f-92fd-a13fe4cb1f95 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19382024-08-28T02:17:11.776ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d0b5e727-ae44-4f55-84e1-f666a4cd8e84 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19392024-08-28T02:17:11.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d0b5e727-ae44-4f55-84e1-f666a4cd8e84 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19402024-08-28T02:17:11.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2087 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d0b5e727-ae44-4f55-84e1-f666a4cd8e84 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19412024-08-28T02:17:12.797ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f5eed577-3ca9-47fa-93ac-e6465debe318 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19422024-08-28T02:17:12.797ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f5eed577-3ca9-47fa-93ac-e6465debe318 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19432024-08-28T02:17:12.799ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2774 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f5eed577-3ca9-47fa-93ac-e6465debe318 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19442024-08-28T02:17:13.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac928842-a761-4c4a-99c8-e82ff97c48f4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19452024-08-28T02:17:13.818ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac928842-a761-4c4a-99c8-e82ff97c48f4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19462024-08-28T02:17:13.820ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac928842-a761-4c4a-99c8-e82ff97c48f4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19472024-08-28T02:17:14.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 62b314bf-f5bf-4276-a127-554aeff36f53 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19482024-08-28T02:17:14.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 62b314bf-f5bf-4276-a127-554aeff36f53 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19492024-08-28T02:17:14.841ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1990 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 62b314bf-f5bf-4276-a127-554aeff36f53 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19502024-08-28T02:17:15.861ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf4b641-4648-47ae-a215-6d17d8cab2e2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19512024-08-28T02:17:15.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf4b641-4648-47ae-a215-6d17d8cab2e2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19522024-08-28T02:17:15.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2246 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf4b641-4648-47ae-a215-6d17d8cab2e2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19532024-08-28T02:17:15.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1960 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = dcf19471-4988-4ec8-b595-862beb97410f response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
19542024-08-28T02:17:16.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2d62e09a-4950-422a-9071-4552d7a7fc60 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19552024-08-28T02:17:16.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2d62e09a-4950-422a-9071-4552d7a7fc60 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19562024-08-28T02:17:16.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2709 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2d62e09a-4950-422a-9071-4552d7a7fc60 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
19572024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
19582024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
19592024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
19602024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19612024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19622024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
19632024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
19642024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19652024-08-28T02:17:17.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
19662024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
19672024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19682024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
19692024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
19702024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19712024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19722024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
19732024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19742024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
19752024-08-28T02:17:17.145ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-28 02:07:17.145354461 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
19762024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
19772024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
19782024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19792024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
19802024-08-28T02:17:17.145ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758
19812024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
19822024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
19832024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.23\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758"}
19842024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
19852024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19862024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
19872024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19882024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
19892024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
19902024-08-28T02:17:17.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
19912024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19922024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
19932024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19942024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
19952024-08-28T02:17:17.145ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
19962024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
19972024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19982024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19992024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
20002024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
20012024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20022024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20032024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
20042024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
20052024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
20062024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
20072024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
20082024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20092024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
20102024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
20112024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
20122024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
20132024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20142024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20152024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20162024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
20172024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20182024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20192024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20202024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20212024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
20222024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20232024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20242024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20252024-08-28T02:17:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20262024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20272024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20282024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20292024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20302024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
20312024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20322024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20332024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20342024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20352024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20362024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20372024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20382024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20392024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20402024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20412024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20422024-08-28T02:17:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("01868911-e8e4-4a77-b105-b83672ce13b2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c854ff-da18-4c2a-9d2f-7e15248c76a8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1245, tv_nsec: 304569461 } })
20432024-08-28T02:17:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6926ac43-4e2e-4378-9bce-42019ec72a30.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1245, tv_nsec: 304591071 } })
20442024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20452024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20462024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20472024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20482024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20492024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
20502024-08-28T02:17:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
20512024-08-28T02:17:17.147ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
20522024-08-28T02:17:17.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.227004ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
20532024-08-28T02:17:17.148ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20542024-08-28T02:17:17.148ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
20552024-08-28T02:17:17.148ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.658102ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
20562024-08-28T02:17:17.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20572024-08-28T02:17:17.150ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.03381ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20582024-08-28T02:17:17.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20592024-08-28T02:17:17.150ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-28 02:07:17.145354461 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
20602024-08-28T02:17:17.150ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.356416ms iteration = 7 status = {"expiration":"2024-08-28T02:07:17.145354461Z","pruned":[]}
20612024-08-28T02:17:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20622024-08-28T02:17:17.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20632024-08-28T02:17:17.152ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20642024-08-28T02:17:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20652024-08-28T02:17:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20662024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20672024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20682024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20692024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20702024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20712024-08-28T02:17:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20722024-08-28T02:17:17.155ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20732024-08-28T02:17:17.157ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1245, tv_nsec: 315410477 } })
20742024-08-28T02:17:17.157ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20752024-08-28T02:17:17.157ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20762024-08-28T02:17:17.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20772024-08-28T02:17:17.158ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1245, tv_nsec: 316717412 } })
20782024-08-28T02:17:17.158ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.906366ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
20792024-08-28T02:17:17.158ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
20802024-08-28T02:17:17.158ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20812024-08-28T02:17:17.158ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20822024-08-28T02:17:17.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20832024-08-28T02:17:17.159ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
20842024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
20852024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
20862024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20872024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
20882024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20892024-08-28T02:17:17.160ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20902024-08-28T02:17:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20912024-08-28T02:17:17.162ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922024-08-28T02:17:17.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20932024-08-28T02:17:17.163ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
20942024-08-28T02:17:17.163ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20952024-08-28T02:17:17.163ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20962024-08-28T02:17:17.163ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20972024-08-28T02:17:17.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20982024-08-28T02:17:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
20992024-08-28T02:17:17.165ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21002024-08-28T02:17:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21012024-08-28T02:17:17.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21022024-08-28T02:17:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21032024-08-28T02:17:17.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21042024-08-28T02:17:17.166ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21052024-08-28T02:17:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21062024-08-28T02:17:17.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21072024-08-28T02:17:17.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21082024-08-28T02:17:17.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21092024-08-28T02:17:17.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21102024-08-28T02:17:17.169ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21112024-08-28T02:17:17.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21122024-08-28T02:17:17.170ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21132024-08-28T02:17:17.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21142024-08-28T02:17:17.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
21152024-08-28T02:17:17.171ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21162024-08-28T02:17:17.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21172024-08-28T02:17:17.173ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1245, tv_nsec: 331543697 } })
21182024-08-28T02:17:17.173ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21192024-08-28T02:17:17.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21202024-08-28T02:17:17.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21212024-08-28T02:17:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21222024-08-28T02:17:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21232024-08-28T02:17:17.174ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
21242024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21252024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21262024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21272024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
21282024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21292024-08-28T02:17:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21302024-08-28T02:17:17.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21312024-08-28T02:17:17.181ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21322024-08-28T02:17:17.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21332024-08-28T02:17:17.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21342024-08-28T02:17:17.184ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21352024-08-28T02:17:17.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21362024-08-28T02:17:17.185ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21372024-08-28T02:17:17.187ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21382024-08-28T02:17:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21392024-08-28T02:17:17.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21402024-08-28T02:17:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21412024-08-28T02:17:17.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21422024-08-28T02:17:17.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
21432024-08-28T02:17:17.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21442024-08-28T02:17:17.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21452024-08-28T02:17:17.210ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.032271ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
21462024-08-28T02:17:17.215ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 70.414328ms iteration = 13 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
21472024-08-28T02:17:17.216ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21482024-08-28T02:17:17.218ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
21492024-08-28T02:17:17.218ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
21502024-08-28T02:17:17.218ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 73.072111ms iteration = 7 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
21512024-08-28T02:17:17.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
21522024-08-28T02:17:17.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21532024-08-28T02:17:17.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21542024-08-28T02:17:17.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21552024-08-28T02:17:17.219ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.332096ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21562024-08-28T02:17:17.221ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.842846ms iteration = 13 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
21572024-08-28T02:17:17.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21582024-08-28T02:17:17.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
21592024-08-28T02:17:17.222ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
21602024-08-28T02:17:17.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
21612024-08-28T02:17:17.223ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21622024-08-28T02:17:17.223ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
21632024-08-28T02:17:17.223ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.614151ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21642024-08-28T02:17:17.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21652024-08-28T02:17:17.224ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21662024-08-28T02:17:17.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21672024-08-28T02:17:17.225ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21682024-08-28T02:17:17.225ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21692024-08-28T02:17:17.225ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21702024-08-28T02:17:17.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21712024-08-28T02:17:17.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21722024-08-28T02:17:17.226ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
21732024-08-28T02:17:17.226ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 80.727123ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
21742024-08-28T02:17:17.227ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 82.486698ms iteration = 4 status = {"nsuccess":5}
21752024-08-28T02:17:17.228ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
21762024-08-28T02:17:17.228ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21772024-08-28T02:17:17.228ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 82.986108ms iteration = 13 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
21782024-08-28T02:17:17.229ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21792024-08-28T02:17:17.229ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21802024-08-28T02:17:17.229ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21812024-08-28T02:17:17.229ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21822024-08-28T02:17:17.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21832024-08-28T02:17:17.231ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21842024-08-28T02:17:17.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21852024-08-28T02:17:17.235ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21862024-08-28T02:17:17.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21872024-08-28T02:17:17.238ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
21882024-08-28T02:17:17.238ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
21892024-08-28T02:17:17.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21902024-08-28T02:17:17.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21912024-08-28T02:17:17.241ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21922024-08-28T02:17:17.241ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21932024-08-28T02:17:17.242ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21942024-08-28T02:17:17.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21952024-08-28T02:17:17.244ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
21962024-08-28T02:17:17.244ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 98.983815ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
21972024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
21982024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21992024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22002024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22012024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22022024-08-28T02:17:17.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22032024-08-28T02:17:17.250ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
22042024-08-28T02:17:17.250ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22052024-08-28T02:17:17.250ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-28 02:11:34.295733 UTC dns_group = internal generation = 1
22062024-08-28T02:17:17.250ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 104.515105ms iteration = 8 status = {"generation":1}
22072024-08-28T02:17:17.256ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22082024-08-28T02:17:17.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22092024-08-28T02:17:17.259ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22102024-08-28T02:17:17.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22112024-08-28T02:17:17.261ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22122024-08-28T02:17:17.261ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 0 version = 2
22132024-08-28T02:17:17.261ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22142024-08-28T02:17:17.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22152024-08-28T02:17:17.263ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
22162024-08-28T02:17:17.263ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 117.86449ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
22172024-08-28T02:17:17.269ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22182024-08-28T02:17:17.275ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22192024-08-28T02:17:17.276ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22202024-08-28T02:17:17.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22212024-08-28T02:17:17.277ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22222024-08-28T02:17:17.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22232024-08-28T02:17:17.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22242024-08-28T02:17:17.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22252024-08-28T02:17:17.282ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22262024-08-28T02:17:17.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22272024-08-28T02:17:17.288ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22282024-08-28T02:17:17.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22292024-08-28T02:17:17.289ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
22302024-08-28T02:17:17.289ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22312024-08-28T02:17:17.289ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d966791a-f4df-48ec-8496-5b5b0300eac7", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22322024-08-28T02:17:17.289ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22332024-08-28T02:17:17.290ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22342024-08-28T02:17:17.291ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22352024-08-28T02:17:17.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22362024-08-28T02:17:17.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22372024-08-28T02:17:17.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22382024-08-28T02:17:17.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22392024-08-28T02:17:17.294ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22402024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22412024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22422024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22432024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22442024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22452024-08-28T02:17:17.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22462024-08-28T02:17:17.297ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
22472024-08-28T02:17:17.297ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22482024-08-28T02:17:17.297ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "199f87e9-f18d-4761-af74-6db0ba207c3b", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22492024-08-28T02:17:17.297ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22502024-08-28T02:17:17.299ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 1 version = 2
22512024-08-28T02:17:17.299ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22522024-08-28T02:17:17.299ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-28 02:11:35.148647 UTC dns_group = external generation = 2
22532024-08-28T02:17:17.299ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 153.597439ms iteration = 8 status = {"generation":2}
22542024-08-28T02:17:17.299ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22552024-08-28T02:17:17.302ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 156.917096ms iteration = 16 status = {"physical_disks_added":0}
22562024-08-28T02:17:17.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22572024-08-28T02:17:17.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22582024-08-28T02:17:17.320ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
22592024-08-28T02:17:17.320ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22602024-08-28T02:17:17.320ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f972421b-e723-4e1f-ba15-91b845c9dd54", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22612024-08-28T02:17:17.320ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22622024-08-28T02:17:17.322ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
22632024-08-28T02:17:17.342ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-28T02:11:15.536604Z, time_modified: 2024-08-28T02:11:15.536604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }) }
22642024-08-28T02:17:17.343ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 197.567074ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]}},"default_endpoint":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
22652024-08-28T02:17:17.494ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22662024-08-28T02:17:17.625ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22672024-08-28T02:17:17.626ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c2fac908-75ed-4db8-9a02-afdfdae7c238", "content-length": "691", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22682024-08-28T02:17:17.626ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
22692024-08-28T02:17:17.626ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
22702024-08-28T02:17:17.626ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
22712024-08-28T02:17:17.627ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22722024-08-28T02:17:17.628ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 482.929369ms iteration = 13 status = {}
22732024-08-28T02:17:17.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22742024-08-28T02:17:17.630ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22752024-08-28T02:17:17.631ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 44d36b70-694e-470e-8f93-9c88e5f413ce roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
22762024-08-28T02:17:17.631ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 44d36b70-694e-470e-8f93-9c88e5f413ce resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
22772024-08-28T02:17:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
22782024-08-28T02:17:17.632ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22792024-08-28T02:17:17.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22802024-08-28T02:17:17.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3403 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 44d36b70-694e-470e-8f93-9c88e5f413ce response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
22812024-08-28T02:17:17.635ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 489.907988ms iteration = 13 status = 0
22822024-08-28T02:17:17.635ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 490.314486ms iteration = 13 status = {}
22832024-08-28T02:17:17.654ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
22842024-08-28T02:17:17.655ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "909d3858-84fc-477f-90fd-c5ef24457109", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22852024-08-28T02:17:17.655ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22862024-08-28T02:17:17.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "36a0f27f-e399-49c5-b10e-458651d4278d", "date": "Wed, 28 Aug 2024 02:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22872024-08-28T02:17:17.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22882024-08-28T02:17:17.658ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
22892024-08-28T02:17:17.658ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 513.227582ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22902024-08-28T02:17:17.660ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22912024-08-28T02:17:17.661ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba0da72-8ffb-4d1f-a174-9e9f437f6602", "content-length": "1", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22922024-08-28T02:17:17.662ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22932024-08-28T02:17:17.662ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "047806d3-7aaa-4444-9abb-aea15c4ff436", "content-length": "372", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
22942024-08-28T02:17:17.662ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22952024-08-28T02:17:17.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22962024-08-28T02:17:17.662ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22972024-08-28T02:17:17.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22982024-08-28T02:17:17.665ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 519.750851ms iteration = 13 status = null
22992024-08-28T02:17:17.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23002024-08-28T02:17:17.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23012024-08-28T02:17:17.666ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23022024-08-28T02:17:17.667ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fcae4961-dd95-423a-a10d-18c61a2a192a", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23032024-08-28T02:17:17.667ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23042024-08-28T02:17:17.668ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23052024-08-28T02:17:17.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23062024-08-28T02:17:17.669ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "71318898-9009-4d81-8c4a-08ad39a339c0", "date": "Wed, 28 Aug 2024 02:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23072024-08-28T02:17:17.669ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23082024-08-28T02:17:17.671ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e5e7d376-9101-4f8d-89bd-3902612b93e8", "date": "Wed, 28 Aug 2024 02:17:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23092024-08-28T02:17:17.671ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23102024-08-28T02:17:17.672ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23112024-08-28T02:17:17.672ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "81b8acb3-96dc-4260-9383-9f2d04574e26", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23122024-08-28T02:17:17.674ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
23132024-08-28T02:17:17.674ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 528.508604ms iteration = 14 status = {}
23142024-08-28T02:17:17.674ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 528.599776ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
23152024-08-28T02:17:17.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23162024-08-28T02:17:17.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23172024-08-28T02:17:17.727ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
23182024-08-28T02:17:17.728ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23192024-08-28T02:17:17.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23202024-08-28T02:17:17.737ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23212024-08-28T02:17:17.737ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "63a2bde8-91f0-48b3-ae3c-a96b5e20163d", "content-length": "121", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23222024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
23232024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
23242024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23252024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23262024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
23272024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
23282024-08-28T02:17:17.737ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
23292024-08-28T02:17:17.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23302024-08-28T02:17:17.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23312024-08-28T02:17:17.739ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23322024-08-28T02:17:17.739ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bd11798a-5ae9-4803-bd3b-83500294c7fc", "content-length": "2", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23332024-08-28T02:17:17.739ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23342024-08-28T02:17:17.739ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9826373d-e7d8-4a4b-b9b6-d0d7f968b245", "content-length": "51", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23352024-08-28T02:17:17.739ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
23362024-08-28T02:17:17.739ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23372024-08-28T02:17:17.830ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e8769c4d-6cb3-4161-8b3f-8e92d187c5a4", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23382024-08-28T02:17:17.830ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
23392024-08-28T02:17:17.830ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23402024-08-28T02:17:17.830ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "821c0707-754a-4864-b3a6-537a358c014e", "content-length": "667", "date": "Wed, 28 Aug 2024 02:17:16 GMT"} })
23412024-08-28T02:17:17.830ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23422024-08-28T02:17:17.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23432024-08-28T02:17:17.834ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23442024-08-28T02:17:17.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23452024-08-28T02:17:17.837ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23462024-08-28T02:17:17.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23472024-08-28T02:17:17.839ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23482024-08-28T02:17:17.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23492024-08-28T02:17:17.846ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 701.576602ms iteration = 13 status = {}
23502024-08-28T02:17:17.932ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8925aed1-049f-4eeb-a5ce-0a8409c6386d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23512024-08-28T02:17:17.932ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8925aed1-049f-4eeb-a5ce-0a8409c6386d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23522024-08-28T02:17:17.934ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2553 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8925aed1-049f-4eeb-a5ce-0a8409c6386d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23532024-08-28T02:17:18.954ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4eb276ed-71f4-47a6-834f-0d6c066339f0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23542024-08-28T02:17:18.954ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4eb276ed-71f4-47a6-834f-0d6c066339f0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23552024-08-28T02:17:18.956ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2894 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4eb276ed-71f4-47a6-834f-0d6c066339f0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23562024-08-28T02:17:19.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7d91f831-7759-454d-a2df-f3e08c78a90f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23572024-08-28T02:17:19.975ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7d91f831-7759-454d-a2df-f3e08c78a90f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23582024-08-28T02:17:19.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7d91f831-7759-454d-a2df-f3e08c78a90f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23592024-08-28T02:17:20.995ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a0d7507-0476-47fc-b7c3-116f17635cce roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23602024-08-28T02:17:20.996ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a0d7507-0476-47fc-b7c3-116f17635cce resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23612024-08-28T02:17:20.997ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2956 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1a0d7507-0476-47fc-b7c3-116f17635cce response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23622024-08-28T02:17:22.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a717ef7f-1449-4917-991e-06a0b5e6b7ca roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23632024-08-28T02:17:22.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a717ef7f-1449-4917-991e-06a0b5e6b7ca resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23642024-08-28T02:17:22.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2588 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a717ef7f-1449-4917-991e-06a0b5e6b7ca response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23652024-08-28T02:17:23.038ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 231060b8-4c34-44e0-a2f7-6e3c0b5133db roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23662024-08-28T02:17:23.039ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 231060b8-4c34-44e0-a2f7-6e3c0b5133db resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23672024-08-28T02:17:23.041ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2408 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 231060b8-4c34-44e0-a2f7-6e3c0b5133db response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23682024-08-28T02:17:24.060ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2df0399f-6c0f-4762-808e-e6713a5edd30 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23692024-08-28T02:17:24.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2df0399f-6c0f-4762-808e-e6713a5edd30 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23702024-08-28T02:17:24.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1965 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2df0399f-6c0f-4762-808e-e6713a5edd30 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23712024-08-28T02:17:25.083ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8fe9d746-5ed9-41d8-8e93-f50bfa573940 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23722024-08-28T02:17:25.083ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8fe9d746-5ed9-41d8-8e93-f50bfa573940 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23732024-08-28T02:17:25.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2343 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8fe9d746-5ed9-41d8-8e93-f50bfa573940 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23742024-08-28T02:17:25.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2015 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = d7575f3b-48bd-4cd2-9d13-6c6d6fa7bdd1 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
23752024-08-28T02:17:26.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92c06879-5009-4006-8866-6f6cfd613634 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23762024-08-28T02:17:26.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92c06879-5009-4006-8866-6f6cfd613634 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23772024-08-28T02:17:26.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3299 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92c06879-5009-4006-8866-6f6cfd613634 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23782024-08-28T02:17:27.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df4a141c-a0d5-4bfd-8101-d0434f9c2a68 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23792024-08-28T02:17:27.126ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df4a141c-a0d5-4bfd-8101-d0434f9c2a68 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23802024-08-28T02:17:27.128ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1863 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df4a141c-a0d5-4bfd-8101-d0434f9c2a68 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
23812024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
23822024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
23832024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23842024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23852024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23862024-08-28T02:17:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23872024-08-28T02:17:27.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23882024-08-28T02:17:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23892024-08-28T02:17:27.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23902024-08-28T02:17:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23912024-08-28T02:17:27.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23922024-08-28T02:17:27.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23932024-08-28T02:17:27.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23942024-08-28T02:17:27.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
23952024-08-28T02:17:27.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23962024-08-28T02:17:27.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23972024-08-28T02:17:27.165ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
23982024-08-28T02:17:27.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.855694ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
23992024-08-28T02:17:27.172ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24002024-08-28T02:17:27.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24012024-08-28T02:17:27.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.128795ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24022024-08-28T02:17:28.146ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6fa87985-d456-48fc-8fcc-70b6660d4d5d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24032024-08-28T02:17:28.146ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6fa87985-d456-48fc-8fcc-70b6660d4d5d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24042024-08-28T02:17:28.148ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1485 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6fa87985-d456-48fc-8fcc-70b6660d4d5d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24052024-08-28T02:17:29.168ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e94aa384-d26a-466c-987f-84fff1df28c3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24062024-08-28T02:17:29.168ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e94aa384-d26a-466c-987f-84fff1df28c3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24072024-08-28T02:17:29.170ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3081 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e94aa384-d26a-466c-987f-84fff1df28c3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24082024-08-28T02:17:30.188ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4e04ef1e-bf38-46b9-8940-4f45e8d16e10 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24092024-08-28T02:17:30.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4e04ef1e-bf38-46b9-8940-4f45e8d16e10 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24102024-08-28T02:17:30.190ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2031 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4e04ef1e-bf38-46b9-8940-4f45e8d16e10 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24112024-08-28T02:17:31.209ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 438b11f4-2c33-49ae-aa22-37fcac95c68a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24122024-08-28T02:17:31.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 438b11f4-2c33-49ae-aa22-37fcac95c68a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24132024-08-28T02:17:31.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2573 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 438b11f4-2c33-49ae-aa22-37fcac95c68a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24142024-08-28T02:17:32.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bda65852-dc20-4dc8-a539-7b3a5d58bafb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24152024-08-28T02:17:32.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bda65852-dc20-4dc8-a539-7b3a5d58bafb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24162024-08-28T02:17:32.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = bda65852-dc20-4dc8-a539-7b3a5d58bafb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24172024-08-28T02:17:33.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7093ac75-dca8-4d11-bdf0-fee78e47f092 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24182024-08-28T02:17:33.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7093ac75-dca8-4d11-bdf0-fee78e47f092 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24192024-08-28T02:17:33.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2071 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7093ac75-dca8-4d11-bdf0-fee78e47f092 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24202024-08-28T02:17:34.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8eb1df18-073a-4feb-b7a1-fc14f1e6130b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24212024-08-28T02:17:34.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8eb1df18-073a-4feb-b7a1-fc14f1e6130b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24222024-08-28T02:17:34.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8eb1df18-073a-4feb-b7a1-fc14f1e6130b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24232024-08-28T02:17:35.292ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99903c8c-fffe-4635-afe7-cf1d6342ff29 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24242024-08-28T02:17:35.292ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99903c8c-fffe-4635-afe7-cf1d6342ff29 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24252024-08-28T02:17:35.294ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2644 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99903c8c-fffe-4635-afe7-cf1d6342ff29 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24262024-08-28T02:17:35.988ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1592 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 04fe63ed-8268-40da-90a3-deabef20a329 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
24272024-08-28T02:17:36.314ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2054bc48-2898-42e6-9339-68a71c51bbb6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24282024-08-28T02:17:36.315ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2054bc48-2898-42e6-9339-68a71c51bbb6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24292024-08-28T02:17:36.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2054bc48-2898-42e6-9339-68a71c51bbb6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24302024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
24312024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
24322024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24332024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24342024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24352024-08-28T02:17:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24362024-08-28T02:17:37.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24372024-08-28T02:17:37.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24382024-08-28T02:17:37.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24392024-08-28T02:17:37.148ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.661683ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24402024-08-28T02:17:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24412024-08-28T02:17:37.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24422024-08-28T02:17:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24432024-08-28T02:17:37.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24442024-08-28T02:17:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24452024-08-28T02:17:37.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24462024-08-28T02:17:37.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
24472024-08-28T02:17:37.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24482024-08-28T02:17:37.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24492024-08-28T02:17:37.163ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
24502024-08-28T02:17:37.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.583869ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
24512024-08-28T02:17:37.337ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fb246730-5da0-4212-b340-f8b9b8b67495 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24522024-08-28T02:17:37.338ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fb246730-5da0-4212-b340-f8b9b8b67495 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24532024-08-28T02:17:37.339ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1663 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fb246730-5da0-4212-b340-f8b9b8b67495 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24542024-08-28T02:17:38.360ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 949fb435-7508-43cd-943a-2e4f95b02e42 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24552024-08-28T02:17:38.361ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 949fb435-7508-43cd-943a-2e4f95b02e42 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24562024-08-28T02:17:38.363ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2294 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 949fb435-7508-43cd-943a-2e4f95b02e42 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24572024-08-28T02:17:39.382ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03705a41-be7e-4ee9-82b4-70360f7acd8f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24582024-08-28T02:17:39.382ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03705a41-be7e-4ee9-82b4-70360f7acd8f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24592024-08-28T02:17:39.384ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1885 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03705a41-be7e-4ee9-82b4-70360f7acd8f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24602024-08-28T02:17:40.405ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf19fa4-ce73-41b7-9543-5d93d400c835 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24612024-08-28T02:17:40.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf19fa4-ce73-41b7-9543-5d93d400c835 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24622024-08-28T02:17:40.407ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2488 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edf19fa4-ce73-41b7-9543-5d93d400c835 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24632024-08-28T02:17:41.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b67c9f2a-9343-473f-be9a-77b207bd5773 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24642024-08-28T02:17:41.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b67c9f2a-9343-473f-be9a-77b207bd5773 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24652024-08-28T02:17:41.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2109 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b67c9f2a-9343-473f-be9a-77b207bd5773 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24662024-08-28T02:17:42.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b24ffad2-5453-4a94-8595-eb1203ab0778 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24672024-08-28T02:17:42.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b24ffad2-5453-4a94-8595-eb1203ab0778 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24682024-08-28T02:17:42.451ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1854 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b24ffad2-5453-4a94-8595-eb1203ab0778 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24692024-08-28T02:17:43.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 56c0d24c-37eb-462e-aa2d-55c559302665 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24702024-08-28T02:17:43.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 56c0d24c-37eb-462e-aa2d-55c559302665 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24712024-08-28T02:17:43.473ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2137 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 56c0d24c-37eb-462e-aa2d-55c559302665 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24722024-08-28T02:17:44.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c43b4fc6-0d93-4cfd-973c-468670ca844c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24732024-08-28T02:17:44.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c43b4fc6-0d93-4cfd-973c-468670ca844c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24742024-08-28T02:17:44.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2232 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c43b4fc6-0d93-4cfd-973c-468670ca844c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24752024-08-28T02:17:44.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7978294d-31a5-404b-ae17-ab7071a1cb8c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/7?limit=100
24762024-08-28T02:17:44.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7978294d-31a5-404b-ae17-ab7071a1cb8c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/7?limit=100
24772024-08-28T02:17:44.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4028 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7978294d-31a5-404b-ae17-ab7071a1cb8c response_code = 200 uri = /nat/ipv4/changeset/7?limit=100
24782024-08-28T02:17:44.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = d1a12925-4d5c-47ab-a9d2-fdc5b6a7345c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/9?limit=100
24792024-08-28T02:17:44.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = d1a12925-4d5c-47ab-a9d2-fdc5b6a7345c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/9?limit=100
24802024-08-28T02:17:44.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3124 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = d1a12925-4d5c-47ab-a9d2-fdc5b6a7345c response_code = 200 uri = /nat/ipv4/changeset/9?limit=100
24812024-08-28T02:17:45.522ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f27406b9-3104-4f3a-bfa8-bc498a7e91fb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24822024-08-28T02:17:45.522ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f27406b9-3104-4f3a-bfa8-bc498a7e91fb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24832024-08-28T02:17:45.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f27406b9-3104-4f3a-bfa8-bc498a7e91fb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24842024-08-28T02:17:45.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1574 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 93b352e5-947a-44a8-a9b1-b95bcc3bbacc response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
24852024-08-28T02:17:46.547ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a3c3578d-8097-4f86-a313-d73da0f2d372 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24862024-08-28T02:17:46.547ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a3c3578d-8097-4f86-a313-d73da0f2d372 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24872024-08-28T02:17:46.550ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3748 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a3c3578d-8097-4f86-a313-d73da0f2d372 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
24882024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
24892024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
24902024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24912024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
24922024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
24932024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24942024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
24952024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24962024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
24972024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24982024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
24992024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
25002024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25012024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25022024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
25032024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
25042024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25052024-08-28T02:17:47.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
25062024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25072024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
25082024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25092024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
25102024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25112024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
25122024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
25132024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25142024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25152024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
25162024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
25172024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
25182024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
25192024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25202024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25212024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25222024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25232024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25242024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25252024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25262024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25272024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25282024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25292024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25302024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25312024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25322024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25332024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
25342024-08-28T02:17:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25352024-08-28T02:17:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25362024-08-28T02:17:47.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25372024-08-28T02:17:47.148ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.703074ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
25382024-08-28T02:17:47.149ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25392024-08-28T02:17:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
25402024-08-28T02:17:47.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25412024-08-28T02:17:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25422024-08-28T02:17:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25432024-08-28T02:17:47.151ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25442024-08-28T02:17:47.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25452024-08-28T02:17:47.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25462024-08-28T02:17:47.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25472024-08-28T02:17:47.157ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25482024-08-28T02:17:47.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25492024-08-28T02:17:47.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25502024-08-28T02:17:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25512024-08-28T02:17:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25522024-08-28T02:17:47.158ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25532024-08-28T02:17:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25542024-08-28T02:17:47.159ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1275, tv_nsec: 318084750 } })
25552024-08-28T02:17:47.159ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1275, tv_nsec: 318274653 } })
25562024-08-28T02:17:47.160ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
25572024-08-28T02:17:47.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25582024-08-28T02:17:47.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25592024-08-28T02:17:47.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25602024-08-28T02:17:47.160ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
25612024-08-28T02:17:47.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25622024-08-28T02:17:47.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25632024-08-28T02:17:47.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25642024-08-28T02:17:47.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25652024-08-28T02:17:47.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25662024-08-28T02:17:47.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25672024-08-28T02:17:47.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25682024-08-28T02:17:47.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25692024-08-28T02:17:47.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25702024-08-28T02:17:47.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25712024-08-28T02:17:47.169ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1275, tv_nsec: 328378414 } })
25722024-08-28T02:17:47.171ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
25732024-08-28T02:17:47.179ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
25742024-08-28T02:17:47.179ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25752024-08-28T02:17:47.179ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25762024-08-28T02:17:47.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25772024-08-28T02:17:47.184ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25782024-08-28T02:17:47.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25792024-08-28T02:17:47.185ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.28745ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25802024-08-28T02:17:47.185ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25812024-08-28T02:17:47.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25822024-08-28T02:17:47.187ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25832024-08-28T02:17:47.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25842024-08-28T02:17:47.188ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 42.672367ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
25852024-08-28T02:17:47.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25862024-08-28T02:17:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25872024-08-28T02:17:47.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25882024-08-28T02:17:47.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25892024-08-28T02:17:47.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25902024-08-28T02:17:47.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
25912024-08-28T02:17:47.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25922024-08-28T02:17:47.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25932024-08-28T02:17:47.194ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 49.383511ms iteration = 14 status = 0
25942024-08-28T02:17:47.195ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25952024-08-28T02:17:47.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25962024-08-28T02:17:47.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25972024-08-28T02:17:47.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25982024-08-28T02:17:47.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25992024-08-28T02:17:47.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26002024-08-28T02:17:47.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26012024-08-28T02:17:47.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26022024-08-28T02:17:47.211ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26032024-08-28T02:17:47.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26042024-08-28T02:17:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26052024-08-28T02:17:47.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
26062024-08-28T02:17:47.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26072024-08-28T02:17:47.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26082024-08-28T02:17:47.215ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
26092024-08-28T02:17:47.215ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
26102024-08-28T02:17:47.215ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 70.639273ms iteration = 14 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
26112024-08-28T02:17:47.220ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
26122024-08-28T02:17:47.220ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.110202ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
26132024-08-28T02:17:47.225ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26142024-08-28T02:17:47.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26152024-08-28T02:17:47.227ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.850136ms iteration = 14 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
26162024-08-28T02:17:47.233ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 87.798983ms iteration = 14 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
26172024-08-28T02:17:47.239ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 94.625749ms iteration = 17 status = {"physical_disks_added":0}
26182024-08-28T02:17:47.249ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26192024-08-28T02:17:47.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26202024-08-28T02:17:47.252ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
26212024-08-28T02:17:47.252ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 107.135337ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
26222024-08-28T02:17:47.293ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
26232024-08-28T02:17:47.293ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26242024-08-28T02:17:47.294ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b04403c8-e9e2-40dc-989a-8cc930aa7026", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
26252024-08-28T02:17:47.294ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26262024-08-28T02:17:47.296ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26272024-08-28T02:17:47.332ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26282024-08-28T02:17:47.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26292024-08-28T02:17:47.334ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26302024-08-28T02:17:47.335ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26312024-08-28T02:17:47.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26322024-08-28T02:17:47.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26332024-08-28T02:17:47.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26342024-08-28T02:17:47.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26352024-08-28T02:17:47.339ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26362024-08-28T02:17:47.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26372024-08-28T02:17:47.354ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26382024-08-28T02:17:47.354ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "567c00a4-22a1-4721-8f78-5ac8a1ee3438", "content-length": "583", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
26392024-08-28T02:17:47.354ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26402024-08-28T02:17:47.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26412024-08-28T02:17:47.356ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26422024-08-28T02:17:47.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26432024-08-28T02:17:47.359ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26442024-08-28T02:17:47.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26452024-08-28T02:17:47.394ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
26462024-08-28T02:17:47.398ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26472024-08-28T02:17:47.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26482024-08-28T02:17:47.435ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
26492024-08-28T02:17:47.435ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26502024-08-28T02:17:47.435ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0e507cec-6a64-405b-8c69-9cc04a5574bd", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
26512024-08-28T02:17:47.435ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26522024-08-28T02:17:47.437ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26532024-08-28T02:17:47.485ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
26542024-08-28T02:17:47.485ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26552024-08-28T02:17:47.485ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5e701e3d-afb9-4a2f-94e7-063367330696", "content-length": "26", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
26562024-08-28T02:17:47.485ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26572024-08-28T02:17:47.486ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26582024-08-28T02:17:47.487ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3aee917a-5e1d-41f6-a033-44f971309470", "content-length": "810", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
26592024-08-28T02:17:47.487ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
26602024-08-28T02:17:47.487ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), vni: Vni(6178120) }} sled = sock
26612024-08-28T02:17:47.487ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26622024-08-28T02:17:47.487ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26632024-08-28T02:17:47.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26642024-08-28T02:17:47.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26652024-08-28T02:17:47.489ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 344.304317ms iteration = 14 status = {}
26662024-08-28T02:17:47.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26672024-08-28T02:17:47.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26682024-08-28T02:17:47.494ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26692024-08-28T02:17:47.494ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c561d007-bc03-466a-9ab8-dfb69f18dcee", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
26702024-08-28T02:17:47.496ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 351.281605ms iteration = 15 status = {}
26712024-08-28T02:17:47.520ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 374.983546ms iteration = 14 status = {}
26722024-08-28T02:17:47.532ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
26732024-08-28T02:17:47.533ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a3dd82fd-cfb0-4524-8afd-806642b4a18b", "content-length": "1", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
26742024-08-28T02:17:47.534ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26752024-08-28T02:17:47.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26762024-08-28T02:17:47.537ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 391.884571ms iteration = 14 status = null
26772024-08-28T02:17:47.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b49e7c5-d716-4fc9-bddf-6a3c6521a541 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
26782024-08-28T02:17:47.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b49e7c5-d716-4fc9-bddf-6a3c6521a541 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
26792024-08-28T02:17:47.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b49e7c5-d716-4fc9-bddf-6a3c6521a541 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
26802024-08-28T02:17:47.609ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26812024-08-28T02:17:47.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26822024-08-28T02:17:47.612ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
26832024-08-28T02:17:47.612ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26842024-08-28T02:17:47.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26852024-08-28T02:17:47.620ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26862024-08-28T02:17:47.620ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "98d19699-4a94-485c-81a3-5cd172ffa212", "content-length": "121", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
26872024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
26882024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
26892024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26902024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26912024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
26922024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
26932024-08-28T02:17:47.621ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
26942024-08-28T02:17:47.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26952024-08-28T02:17:47.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26962024-08-28T02:17:47.623ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26972024-08-28T02:17:47.624ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "659e06a6-c095-4073-9f2b-23fc35e135b2", "content-length": "2", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
26982024-08-28T02:17:47.624ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26992024-08-28T02:17:47.624ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "084f4557-ff61-43bf-acd3-63f892cc427b", "content-length": "51", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
27002024-08-28T02:17:47.624ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
27012024-08-28T02:17:47.624ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27022024-08-28T02:17:47.721ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7d0bda70-c070-4082-90df-61340755e9d7", "date": "Wed, 28 Aug 2024 02:17:47 GMT"} })
27032024-08-28T02:17:47.721ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
27042024-08-28T02:17:47.721ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27052024-08-28T02:17:47.725ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "42ada670-d85f-4328-97ac-7679d29ecf41", "content-length": "667", "date": "Wed, 28 Aug 2024 02:17:46 GMT"} })
27062024-08-28T02:17:47.725ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27072024-08-28T02:17:47.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27082024-08-28T02:17:47.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27092024-08-28T02:17:47.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27102024-08-28T02:17:47.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27112024-08-28T02:17:47.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27122024-08-28T02:17:47.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27132024-08-28T02:17:47.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27142024-08-28T02:17:47.743ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 598.499844ms iteration = 14 status = {}
27152024-08-28T02:17:48.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3e5b3310-9ca0-437a-be9c-45e9397a9ef0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27162024-08-28T02:17:48.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3e5b3310-9ca0-437a-be9c-45e9397a9ef0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27172024-08-28T02:17:48.602ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1877 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3e5b3310-9ca0-437a-be9c-45e9397a9ef0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27182024-08-28T02:17:49.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1730c0f8-6f52-4f3c-812d-d4c0cfdf0935 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27192024-08-28T02:17:49.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1730c0f8-6f52-4f3c-812d-d4c0cfdf0935 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27202024-08-28T02:17:49.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1730c0f8-6f52-4f3c-812d-d4c0cfdf0935 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27212024-08-28T02:17:50.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f7e57010-3f2f-4877-a6fa-fc8e6fffd3ab roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27222024-08-28T02:17:50.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f7e57010-3f2f-4877-a6fa-fc8e6fffd3ab resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27232024-08-28T02:17:50.644ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2210 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f7e57010-3f2f-4877-a6fa-fc8e6fffd3ab response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27242024-08-28T02:17:51.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea441a52-5dd1-4658-931d-8699ce1625e3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27252024-08-28T02:17:51.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea441a52-5dd1-4658-931d-8699ce1625e3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27262024-08-28T02:17:51.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ea441a52-5dd1-4658-931d-8699ce1625e3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27272024-08-28T02:17:51.697ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "628daf21-533a-4de6-97a8-a7a705fdd40c", "content-length": "128", "date": "Wed, 28 Aug 2024 02:17:51 GMT"} })
27282024-08-28T02:17:51.697ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 state = Starting
27292024-08-28T02:17:51.700ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1853 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T02:17:43.923233Z }
27302024-08-28T02:17:51.700ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 roles = RoleSet { roles: {} }
27312024-08-28T02:17:51.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 resource = Database result = Ok(())
27322024-08-28T02:17:51.706ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1853 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69b53f00-d1e9-446d-8edf-95990d54f4cf uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T02:17:51.698662477Z }
27332024-08-28T02:17:51.706ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69b53f00-d1e9-446d-8edf-95990d54f4cf roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27342024-08-28T02:17:51.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69b53f00-d1e9-446d-8edf-95990d54f4cf resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27352024-08-28T02:17:51.733ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
27362024-08-28T02:17:51.733ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 4.588596785s iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
27372024-08-28T02:17:51.742ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1:0]:46971
27382024-08-28T02:17:51.742ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:46971 req_id = 741b2aa2-e255-4fe6-84e6-343692c7f267 roles = RoleSet { roles: {} } uri = /metrics/producers
27392024-08-28T02:17:51.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:46971 req_id = 741b2aa2-e255-4fe6-84e6-343692c7f267 resource = Database result = Ok(()) uri = /metrics/producers
27402024-08-28T02:17:51.745ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 46920 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69b53f00-d1e9-446d-8edf-95990d54f4cf response_code = 204 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27412024-08-28T02:17:51.747ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1853 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce7e3623-8d94-40a0-9f64-42b5d24828d8 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T02:17:51.746880984Z }
27422024-08-28T02:17:51.747ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce7e3623-8d94-40a0-9f64-42b5d24828d8 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27432024-08-28T02:17:51.747ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce7e3623-8d94-40a0-9f64-42b5d24828d8 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27442024-08-28T02:17:51.775ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
27452024-08-28T02:17:51.775ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:46971 req_id = 741b2aa2-e255-4fe6-84e6-343692c7f267 roles = RoleSet { roles: {} } uri = /metrics/producers
27462024-08-28T02:17:51.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:46971 req_id = 741b2aa2-e255-4fe6-84e6-343692c7f267 resource = Database result = Ok(()) uri = /metrics/producers
27472024-08-28T02:17:51.780ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 33144 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce7e3623-8d94-40a0-9f64-42b5d24828d8 response_code = 204 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
27482024-08-28T02:17:51.805ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
27492024-08-28T02:17:51.805ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "28869f4b-aa1b-4705-be08-dfda4d002d59", "date": "Wed, 28 Aug 2024 02:17:51 GMT"} })
27502024-08-28T02:17:51.805ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4
27512024-08-28T02:17:51.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 64178 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:46971 req_id = 741b2aa2-e255-4fe6-84e6-343692c7f267 response_code = 201 uri = /metrics/producers
27522024-08-28T02:17:52.687ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8ca529bd-7630-4c13-9697-29870f5d2680 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27532024-08-28T02:17:52.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8ca529bd-7630-4c13-9697-29870f5d2680 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27542024-08-28T02:17:52.689ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2721 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8ca529bd-7630-4c13-9697-29870f5d2680 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27552024-08-28T02:17:53.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 07d131bc-9a3a-4764-82f5-bbd1cb96bce6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27562024-08-28T02:17:53.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 07d131bc-9a3a-4764-82f5-bbd1cb96bce6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27572024-08-28T02:17:53.710ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1947 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 07d131bc-9a3a-4764-82f5-bbd1cb96bce6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27582024-08-28T02:17:54.730ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 271a94b2-186d-4b23-8eb5-45f8c4cb013a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27592024-08-28T02:17:54.731ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 271a94b2-186d-4b23-8eb5-45f8c4cb013a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27602024-08-28T02:17:54.732ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2366 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 271a94b2-186d-4b23-8eb5-45f8c4cb013a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27612024-08-28T02:17:55.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92aa1cfb-e3ff-413e-92f9-ff3f38008678 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27622024-08-28T02:17:55.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92aa1cfb-e3ff-413e-92f9-ff3f38008678 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27632024-08-28T02:17:55.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 92aa1cfb-e3ff-413e-92f9-ff3f38008678 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27642024-08-28T02:17:55.992ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4700 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 47223661-44c1-4de3-8e3c-d527564f0f37 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
27652024-08-28T02:17:56.774ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bf00f8-ebe0-4475-8667-9d9048427370 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27662024-08-28T02:17:56.775ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bf00f8-ebe0-4475-8667-9d9048427370 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27672024-08-28T02:17:56.777ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2453 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5bf00f8-ebe0-4475-8667-9d9048427370 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27682024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
27692024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27702024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
27712024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27722024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27732024-08-28T02:17:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27742024-08-28T02:17:57.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27752024-08-28T02:17:57.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27762024-08-28T02:17:57.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27772024-08-28T02:17:57.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27782024-08-28T02:17:57.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27792024-08-28T02:17:57.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27802024-08-28T02:17:57.151ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.623422ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27812024-08-28T02:17:57.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27822024-08-28T02:17:57.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27832024-08-28T02:17:57.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27842024-08-28T02:17:57.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
27852024-08-28T02:17:57.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27862024-08-28T02:17:57.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27872024-08-28T02:17:57.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
27882024-08-28T02:17:57.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.265792ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
27892024-08-28T02:17:57.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12d3fa63-ba72-47cd-8099-5200ad7bda4d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27902024-08-28T02:17:57.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12d3fa63-ba72-47cd-8099-5200ad7bda4d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27912024-08-28T02:17:57.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2006 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 12d3fa63-ba72-47cd-8099-5200ad7bda4d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27922024-08-28T02:17:58.817ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99e43400-eca2-4102-9fe7-d583cec55ffa roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27932024-08-28T02:17:58.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99e43400-eca2-4102-9fe7-d583cec55ffa resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27942024-08-28T02:17:58.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3004 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 99e43400-eca2-4102-9fe7-d583cec55ffa response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27952024-08-28T02:17:59.838ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e55cd65-6a39-44e6-8cc2-6ae0a7a62ef8 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27962024-08-28T02:17:59.839ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e55cd65-6a39-44e6-8cc2-6ae0a7a62ef8 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27972024-08-28T02:17:59.841ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2562 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e55cd65-6a39-44e6-8cc2-6ae0a7a62ef8 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27982024-08-28T02:18:00.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 776aa182-3450-42a9-8978-feed35d70ac8 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
27992024-08-28T02:18:00.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 776aa182-3450-42a9-8978-feed35d70ac8 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28002024-08-28T02:18:00.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2925 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 776aa182-3450-42a9-8978-feed35d70ac8 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28012024-08-28T02:18:01.882ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 757c4574-343d-4798-aea3-e7f085b9b586 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28022024-08-28T02:18:01.882ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 757c4574-343d-4798-aea3-e7f085b9b586 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28032024-08-28T02:18:01.884ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2308 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 757c4574-343d-4798-aea3-e7f085b9b586 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28042024-08-28T02:18:02.037ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1853 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24dd6f20-3552-4108-9248-13acafd6e282 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T02:18:02.036931604Z }
28052024-08-28T02:18:02.037ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24dd6f20-3552-4108-9248-13acafd6e282 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28062024-08-28T02:18:02.037ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24dd6f20-3552-4108-9248-13acafd6e282 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28072024-08-28T02:18:02.045ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8718 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 24dd6f20-3552-4108-9248-13acafd6e282 response_code = 204 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28082024-08-28T02:18:02.689ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = bcf2e984-11dc-4943-90b2-9586d52feed5 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/9?limit=100
28092024-08-28T02:18:02.690ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = bcf2e984-11dc-4943-90b2-9586d52feed5 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/9?limit=100
28102024-08-28T02:18:02.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4700 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = bcf2e984-11dc-4943-90b2-9586d52feed5 response_code = 200 uri = /nat/ipv4/changeset/9?limit=100
28112024-08-28T02:18:02.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 8387ae2b-d0b5-4197-a4c3-5ecc11a79501 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
28122024-08-28T02:18:02.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 8387ae2b-d0b5-4197-a4c3-5ecc11a79501 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
28132024-08-28T02:18:02.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3941 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 8387ae2b-d0b5-4197-a4c3-5ecc11a79501 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
28142024-08-28T02:18:02.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 667ce472-2099-4e7a-92a3-76d722accf12 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28152024-08-28T02:18:02.905ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 667ce472-2099-4e7a-92a3-76d722accf12 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28162024-08-28T02:18:02.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2133 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 667ce472-2099-4e7a-92a3-76d722accf12 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28172024-08-28T02:18:03.699ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 689dbea0-1fa9-4807-8453-e196c6ef1d11 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
28182024-08-28T02:18:03.699ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 689dbea0-1fa9-4807-8453-e196c6ef1d11 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
28192024-08-28T02:18:03.706ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7065 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 689dbea0-1fa9-4807-8453-e196c6ef1d11 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
28202024-08-28T02:18:03.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db294c53-af14-4ed8-844d-cd5b8f7cf189 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28212024-08-28T02:18:03.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db294c53-af14-4ed8-844d-cd5b8f7cf189 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28222024-08-28T02:18:03.927ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1592 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = db294c53-af14-4ed8-844d-cd5b8f7cf189 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
28232024-08-28T02:18:04.426ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1853 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T02:18:04.425617066Z }
28242024-08-28T02:18:04.426ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28252024-08-28T02:18:04.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28262024-08-28T02:18:04.461ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_needs_update = true
28272024-08-28T02:18:04.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28282024-08-28T02:18:04.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28292024-08-28T02:18:04.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28302024-08-28T02:18:04.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28312024-08-28T02:18:04.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28322024-08-28T02:18:04.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28332024-08-28T02:18:04.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28342024-08-28T02:18:04.468ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28352024-08-28T02:18:04.469ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28362024-08-28T02:18:04.469ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28372024-08-28T02:18:04.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28382024-08-28T02:18:04.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28392024-08-28T02:18:04.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28402024-08-28T02:18:04.474ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bb0007f2-a695-4aac-b57b-8cca99a33c5b, lookup_type: ById(bb0007f2-a695-4aac-b57b-8cca99a33c5b) }, key: 286513bd-9597-457f-90e3-0d7965df272f, lookup_type: ById(286513bd-9597-457f-90e3-0d7965df272f) }, key: 29444af2-b828-4c13-9cb7-17c7a56ed0e4, lookup_type: ById(29444af2-b828-4c13-9cb7-17c7a56ed0e4) } result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28412024-08-28T02:18:04.477ZINFOnexus (dropshot_internal): starting update saga for 29444af2-b828-4c13-9cb7-17c7a56ed0e4 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1349 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T02:18:04.425617066Z }
28422024-08-28T02:18:04.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 51456 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b6009f85-a0ee-4246-99b8-a43df88ad8f5 response_code = 204 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28432024-08-28T02:18:04.477ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:212 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update
28442024-08-28T02:18:04.477ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"29444af2-b828-4c13-9cb7-17c7a56ed0e4","lookup_type":{"ById":"29444af2-b828-4c13-9cb7-17c7a56ed0e4"},"parent":{"key":"286513bd-9597-457f-90e3-0d7965df272f","lookup_type":{"ById":"286513bd-9597-457f-90e3-0d7965df272f"},"parent":{"key":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","lookup_type":{"ById":"bb0007f2-a695-4aac-b57b-8cca99a33c5b"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28452024-08-28T02:18:04.477ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update
28462024-08-28T02:18:04.477ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1853 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 914a350e-58eb-4a04-ad0e-e10971aec8e3 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T02:18:04.425617066Z }
28472024-08-28T02:18:04.477ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 914a350e-58eb-4a04-ad0e-e10971aec8e3 roles = RoleSet { roles: {} } uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28482024-08-28T02:18:04.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 914a350e-58eb-4a04-ad0e-e10971aec8e3 resource = Database result = Ok(()) uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28492024-08-28T02:18:04.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3723 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 914a350e-58eb-4a04-ad0e-e10971aec8e3 response_code = 204 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
28502024-08-28T02:18:04.504ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:310 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update
28512024-08-28T02:18:04.504ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28522024-08-28T02:18:04.504ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28532024-08-28T02:18:04.504ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28542024-08-28T02:18:04.508ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28552024-08-28T02:18:04.508ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28562024-08-28T02:18:04.512ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28572024-08-28T02:18:04.512ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28582024-08-28T02:18:04.515ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28592024-08-28T02:18:04.515ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("a66250b7-479c-4e87-89dc-64351cfede8b")) node_id = 0 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28602024-08-28T02:18:04.519ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28612024-08-28T02:18:04.519ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28622024-08-28T02:18:04.527ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update
28632024-08-28T02:18:04.527ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28642024-08-28T02:18:04.528ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28652024-08-28T02:18:04.530ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28662024-08-28T02:18:04.530ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28672024-08-28T02:18:04.532ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28682024-08-28T02:18:04.532ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28692024-08-28T02:18:04.534ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28702024-08-28T02:18:04.534ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28712024-08-28T02:18:04.536ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28722024-08-28T02:18:04.536ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28732024-08-28T02:18:04.537ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28742024-08-28T02:18:04.537ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28752024-08-28T02:18:04.539ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28762024-08-28T02:18:04.541ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bb0007f2-a695-4aac-b57b-8cca99a33c5b, lookup_type: ById(bb0007f2-a695-4aac-b57b-8cca99a33c5b) }, key: 286513bd-9597-457f-90e3-0d7965df272f, lookup_type: ById(286513bd-9597-457f-90e3-0d7965df272f) }, key: 29444af2-b828-4c13-9cb7-17c7a56ed0e4, lookup_type: ById(29444af2-b828-4c13-9cb7-17c7a56ed0e4) } result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28772024-08-28T02:18:04.541ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance updater_id = a66250b7-479c-4e87-89dc-64351cfede8b
28782024-08-28T02:18:04.542ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28792024-08-28T02:18:04.542ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28802024-08-28T02:18:04.550ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance updater_id = a66250b7-479c-4e87-89dc-64351cfede8b
28812024-08-28T02:18:04.550ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28822024-08-28T02:18:04.550ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance
28832024-08-28T02:18:04.554ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 locked_gen = Generation(Generation(2)) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = LockInstance updater_id = a66250b7-479c-4e87-89dc-64351cfede8b
28842024-08-28T02:18:04.557ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28852024-08-28T02:18:04.557ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("a66250b7-479c-4e87-89dc-64351cfede8b")}) node_id = 1 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28862024-08-28T02:18:04.561ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
28872024-08-28T02:18:04.561ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 947cfee1-faed-454a-8a9a-541767457944
28882024-08-28T02:18:04.564ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28892024-08-28T02:18:04.564ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28902024-08-28T02:18:04.565ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28912024-08-28T02:18:04.566ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28922024-08-28T02:18:04.567ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28932024-08-28T02:18:04.567ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28942024-08-28T02:18:04.568ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28952024-08-28T02:18:04.571ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bb0007f2-a695-4aac-b57b-8cca99a33c5b, lookup_type: ById(bb0007f2-a695-4aac-b57b-8cca99a33c5b) }, key: 286513bd-9597-457f-90e3-0d7965df272f, lookup_type: ById(286513bd-9597-457f-90e3-0d7965df272f) }, key: 29444af2-b828-4c13-9cb7-17c7a56ed0e4, lookup_type: ById(29444af2-b828-4c13-9cb7-17c7a56ed0e4) } result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28962024-08-28T02:18:04.571ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28972024-08-28T02:18:04.571ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
28982024-08-28T02:18:04.575ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 8b43f99c-a0e7-445b-8ea1-1ee3b968f594, time_created: 2024-08-28T02:17:43.923233Z, time_deleted: None, instance_id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4, sled_id: 40b49400-5cd0-4d5c-a006-b241a1ca8261, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-08-28T02:18:04.425617Z, gen: Generation(Generation(7)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-08-28T02:17:43.969444Z, gen: Generation(Generation(3)), propolis_id: Some(8b43f99c-a0e7-445b-8ea1-1ee3b968f594), dst_propolis_id: None, migration_id: None, nexus_state: Vmm } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(8b43f99c-a0e7-445b-8ea1-1ee3b968f594 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-08-28T02:18:04.575917511Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm }
28992024-08-28T02:18:04.575ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:212 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29002024-08-28T02:18:04.578ZINFOnexus (SEC): saga create dag = {"end_node":13,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[12,0,null],[11,13,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"29444af2-b828-4c13-9cb7-17c7a56ed0e4","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"8b43f99c-a0e7-445b-8ea1-1ee3b968f594"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Start":{"params":{"authz_instance":{"key":"29444af2-b828-4c13-9cb7-17c7a56ed0e4","lookup_type":{"ById":"29444af2-b828-4c13-9cb7-17c7a56ed0e4"},"parent":{"key":"286513bd-9597-457f-90e3-0d7965df272f","lookup_type":{"ById":"286513bd-9597-457f-90e3-0d7965df272f"},"parent":{"key":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","lookup_type":{"ById":"bb0007f2-a695-4aac-b57b-8cca99a33c5b"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"a66250b7-479c-4e87-89dc-64351cfede8b"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"286513bd-9597-457f-90e3-0d7965df272f","ram_diff":1073741824},"destroy_active_vmm":"8b43f99c-a0e7-445b-8ea1-1ee3b968f594","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_updated":"2024-08-28T02:18:04.575917511Z"}}}}},"End"]},"saga_name":"instance-update","start_node":12} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29012024-08-28T02:18:04.578ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29022024-08-28T02:18:04.607ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:310 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29032024-08-28T02:18:04.607ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29042024-08-28T02:18:04.607ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29052024-08-28T02:18:04.607ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29062024-08-28T02:18:04.611ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29072024-08-28T02:18:04.611ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29082024-08-28T02:18:04.614ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29092024-08-28T02:18:04.614ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29102024-08-28T02:18:04.617ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29112024-08-28T02:18:04.617ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("4ee77311-1c23-485d-b944-327e67a90149")) node_id = 0 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29122024-08-28T02:18:04.621ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29132024-08-28T02:18:04.621ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29142024-08-28T02:18:04.626ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 parent_lock = UpdaterLock { updater_id: a66250b7-479c-4e87-89dc-64351cfede8b, locked_gen: Generation(Generation(2)) } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29152024-08-28T02:18:04.626ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = BecomeUpdater
29162024-08-28T02:18:04.627ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = BecomeUpdater
29172024-08-28T02:18:04.640ZDEBGnexus (SagaExecutor): inherited lock from a66250b7-479c-4e87-89dc-64351cfede8b to 4ee77311-1c23-485d-b944-327e67a90149 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = a66250b7-479c-4e87-89dc-64351cfede8b saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = BecomeUpdater updater_id = 4ee77311-1c23-485d-b944-327e67a90149
29182024-08-28T02:18:04.640ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:932 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29192024-08-28T02:18:04.640ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29202024-08-28T02:18:04.640ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("4ee77311-1c23-485d-b944-327e67a90149")}) node_id = 1 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29212024-08-28T02:18:04.643ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29222024-08-28T02:18:04.643ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29232024-08-28T02:18:04.646ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29242024-08-28T02:18:04.646ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29252024-08-28T02:18:04.650ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29262024-08-28T02:18:04.650ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29272024-08-28T02:18:04.653ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:985 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29282024-08-28T02:18:04.656ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4
29292024-08-28T02:18:04.658ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29302024-08-28T02:18:04.658ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29312024-08-28T02:18:04.663ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29322024-08-28T02:18:04.663ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29332024-08-28T02:18:04.665ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
29342024-08-28T02:18:04.665ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29352024-08-28T02:18:04.666ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UpdateNetworkConfig
29362024-08-28T02:18:04.668ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
29372024-08-28T02:18:04.668ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
29382024-08-28T02:18:04.668ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
29392024-08-28T02:18:04.670ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
29402024-08-28T02:18:04.671ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
29412024-08-28T02:18:04.671ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
29422024-08-28T02:18:04.671ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
29432024-08-28T02:18:04.679ZDEBGnexus: notifying dendrite of updates instance_id = Some(29444af2-b828-4c13-9cb7-17c7a56ed0e4 (instance)) switch = switch0
29442024-08-28T02:18:04.682ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 840896340 } })
29452024-08-28T02:18:04.682ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1050
29462024-08-28T02:18:04.713ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
29472024-08-28T02:18:04.713ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29482024-08-28T02:18:04.713ZDEBGnexus (MgsClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "41c91a87-af26-4a77-b91e-e8178cbc5de1", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:04 GMT"} })
29492024-08-28T02:18:04.713ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29502024-08-28T02:18:04.714ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29512024-08-28T02:18:04.743ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
29522024-08-28T02:18:04.744ZDEBGnexus (DpdClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "41f7b316-3c7b-40f5-8f81-e4ffd1044019", "content-length": "4", "date": "Wed, 28 Aug 2024 02:18:03 GMT"} })
29532024-08-28T02:18:04.746ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29542024-08-28T02:18:04.746ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29552024-08-28T02:18:04.751ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29562024-08-28T02:18:04.751ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29572024-08-28T02:18:04.754ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
29582024-08-28T02:18:04.754ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
29592024-08-28T02:18:04.766ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1060 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-08-28T02:18:04.757477Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 286513bd-9597-457f-90e3-0d7965df272f, time_modified: Some(2024-08-28T02:18:04.757477Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, time_modified: Some(2024-08-28T02:18:04.757477Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29602024-08-28T02:18:04.766ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29612024-08-28T02:18:04.766ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29622024-08-28T02:18:04.770ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29632024-08-28T02:18:04.770ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29642024-08-28T02:18:04.774ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1095 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29652024-08-28T02:18:04.777ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UnassignOximeterProducer
29662024-08-28T02:18:04.777ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UnassignOximeterProducer
29672024-08-28T02:18:04.787ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 producer_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29682024-08-28T02:18:04.787ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UnassignOximeterProducer
29692024-08-28T02:18:04.787ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = UnassignOximeterProducer
29702024-08-28T02:18:04.820ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29712024-08-28T02:18:04.820ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/29444af2-b828-4c13-9cb7-17c7a56ed0e4
29722024-08-28T02:18:04.824ZDEBGnexus (SagaExecutor): client response oximeter-collector = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers/29444af2-b828-4c13-9cb7-17c7a56ed0e4", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cfbd1bbd-ed97-4390-aba8-605e99caa970", "date": "Wed, 28 Aug 2024 02:18:04 GMT"} }) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29732024-08-28T02:18:04.824ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 producer_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29742024-08-28T02:18:04.825ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29752024-08-28T02:18:04.825ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29762024-08-28T02:18:04.829ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
29772024-08-28T02:18:04.829ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
29782024-08-28T02:18:04.832ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 lock = UpdaterLock { updater_id: 4ee77311-1c23-485d-b944-327e67a90149, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-08-28T02:18:04.575917511Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29792024-08-28T02:18:04.832ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29802024-08-28T02:18:04.833ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29812024-08-28T02:18:04.871ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1144 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 new_runtime = InstanceRuntimeState { time_updated: 2024-08-28T02:18:04.575917511Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
29822024-08-28T02:18:04.871ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29832024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Signaled
29842024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29852024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29862024-08-28T02:18:04.871ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29872024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Signaled
29882024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29892024-08-28T02:18:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29902024-08-28T02:18:04.873ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29912024-08-28T02:18:04.873ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29922024-08-28T02:18:04.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932024-08-28T02:18:04.874ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29942024-08-28T02:18:04.875ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29952024-08-28T02:18:04.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29962024-08-28T02:18:04.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29972024-08-28T02:18:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29982024-08-28T02:18:04.876ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
29992024-08-28T02:18:04.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30002024-08-28T02:18:04.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30012024-08-28T02:18:04.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30022024-08-28T02:18:04.879ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: bb0007f2-a695-4aac-b57b-8cca99a33c5b, lookup_type: ById(bb0007f2-a695-4aac-b57b-8cca99a33c5b) }, key: 286513bd-9597-457f-90e3-0d7965df272f, lookup_type: ById(286513bd-9597-457f-90e3-0d7965df272f) }, key: 29444af2-b828-4c13-9cb7-17c7a56ed0e4, lookup_type: ById(29444af2-b828-4c13-9cb7-17c7a56ed0e4) } result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
30032024-08-28T02:18:04.879ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
30042024-08-28T02:18:04.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30052024-08-28T02:18:04.880ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = CommitInstanceUpdates
30062024-08-28T02:18:04.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30072024-08-28T02:18:04.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30082024-08-28T02:18:04.885ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30092024-08-28T02:18:04.885ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30102024-08-28T02:18:04.889ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30112024-08-28T02:18:04.889ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30122024-08-28T02:18:04.890ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30132024-08-28T02:18:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30142024-08-28T02:18:04.893ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30152024-08-28T02:18:04.893ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("29444af2-b828-4c13-9cb7-17c7a56ed0e4"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("8b43f99c-a0e7-445b-8ea1-1ee3b968f594")}) node_id = 7 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30162024-08-28T02:18:04.912ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30172024-08-28T02:18:04.913ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "71097645-4e1f-4ba4-8270-738d6716c48c", "content-length": "372", "date": "Wed, 28 Aug 2024 02:18:04 GMT"} })
30182024-08-28T02:18:04.913ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30192024-08-28T02:18:04.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30202024-08-28T02:18:04.915ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30212024-08-28T02:18:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30222024-08-28T02:18:04.917ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30232024-08-28T02:18:04.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30242024-08-28T02:18:04.917ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30252024-08-28T02:18:04.917ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30262024-08-28T02:18:04.919ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30272024-08-28T02:18:04.919ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ad12e7e6-83f7-4509-9c01-e3d786bdbbab", "date": "Wed, 28 Aug 2024 02:18:04 GMT"} })
30282024-08-28T02:18:04.920ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30292024-08-28T02:18:04.920ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30302024-08-28T02:18:04.921ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 50.034994ms iteration = 16 status = {}
30312024-08-28T02:18:04.924ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30322024-08-28T02:18:04.924ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30332024-08-28T02:18:04.927ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
30342024-08-28T02:18:04.927ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = ReleaseSledResources
30352024-08-28T02:18:04.927ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = ReleaseSledResources
30362024-08-28T02:18:04.938ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30372024-08-28T02:18:04.938ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30382024-08-28T02:18:04.945ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30392024-08-28T02:18:04.945ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30402024-08-28T02:18:04.946ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412024-08-28T02:18:04.948ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30422024-08-28T02:18:04.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30432024-08-28T02:18:04.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30442024-08-28T02:18:04.949ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = be174520-35a9-4796-8640-07dbeceb3aea roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30452024-08-28T02:18:04.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = be174520-35a9-4796-8640-07dbeceb3aea resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30462024-08-28T02:18:04.950ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30472024-08-28T02:18:04.951ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update
30482024-08-28T02:18:04.951ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = MarkVmmDeleted
30492024-08-28T02:18:04.951ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2543 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = be174520-35a9-4796-8640-07dbeceb3aea response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30502024-08-28T02:18:04.951ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_node = MarkVmmDeleted
30512024-08-28T02:18:04.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30522024-08-28T02:18:04.952ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30532024-08-28T02:18:04.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30542024-08-28T02:18:04.960ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30552024-08-28T02:18:04.960ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30562024-08-28T02:18:04.963ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30572024-08-28T02:18:04.963ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30582024-08-28T02:18:04.966ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30592024-08-28T02:18:04.966ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30602024-08-28T02:18:04.970ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30612024-08-28T02:18:04.970ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30622024-08-28T02:18:04.973ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30632024-08-28T02:18:04.973ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30642024-08-28T02:18:04.986ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30652024-08-28T02:18:04.986ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "81b1484d-69b6-4b9d-868d-f9a3bf3a3e5b", "content-length": "691", "date": "Wed, 28 Aug 2024 02:18:04 GMT"} })
30662024-08-28T02:18:04.986ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
30672024-08-28T02:18:04.986ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
30682024-08-28T02:18:04.986ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
30692024-08-28T02:18:04.989ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 118.129816ms iteration = 15 status = {}
30702024-08-28T02:18:04.990ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30712024-08-28T02:18:04.990ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06
30722024-08-28T02:18:05.022ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30732024-08-28T02:18:05.022ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:361 saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 saga_name = instance-update saga_result = SagaResult { saga_id: 0f3693ca-3397-4fae-ae01-22d311bcdb06, saga_log: SagaLog { saga_id: 0f3693ca-3397-4fae-ae01-22d311bcdb06, unwinding: false, events: [N012 started, N012 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N013 started, N013 succeeded], node_status: {0: Succeeded(String("4ee77311-1c23-485d-b944-327e67a90149")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("4ee77311-1c23-485d-b944-327e67a90149")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("29444af2-b828-4c13-9cb7-17c7a56ed0e4"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("8b43f99c-a0e7-445b-8ea1-1ee3b968f594")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("29444af2-b828-4c13-9cb7-17c7a56ed0e4"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("8b43f99c-a0e7-445b-8ea1-1ee3b968f594")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("4ee77311-1c23-485d-b944-327e67a90149"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("4ee77311-1c23-485d-b944-327e67a90149")}} }) }
30742024-08-28T02:18:05.022ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 0f3693ca-3397-4fae-ae01-22d311bcdb06 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update
30752024-08-28T02:18:05.022ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30762024-08-28T02:18:05.022ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 947cfee1-faed-454a-8a9a-541767457944
30772024-08-28T02:18:05.026ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30782024-08-28T02:18:05.026ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 947cfee1-faed-454a-8a9a-541767457944
30792024-08-28T02:18:05.029ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30802024-08-28T02:18:05.029ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 947cfee1-faed-454a-8a9a-541767457944
30812024-08-28T02:18:05.033ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 947cfee1-faed-454a-8a9a-541767457944 sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30822024-08-28T02:18:05.033ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 947cfee1-faed-454a-8a9a-541767457944
30832024-08-28T02:18:05.068ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update sec_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
30842024-08-28T02:18:05.068ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:361 saga_id = 947cfee1-faed-454a-8a9a-541767457944 saga_name = start-instance-update saga_result = SagaResult { saga_id: 947cfee1-faed-454a-8a9a-541767457944, saga_log: SagaLog { saga_id: 947cfee1-faed-454a-8a9a-541767457944, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("a66250b7-479c-4e87-89dc-64351cfede8b")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("a66250b7-479c-4e87-89dc-64351cfede8b")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("a66250b7-479c-4e87-89dc-64351cfede8b"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("a66250b7-479c-4e87-89dc-64351cfede8b")}} }) }
30852024-08-28T02:18:05.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 1e30b47a-076b-4304-90d3-21a196428666 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
30862024-08-28T02:18:05.709ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 1e30b47a-076b-4304-90d3-21a196428666 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
30872024-08-28T02:18:05.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3859 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 1e30b47a-076b-4304-90d3-21a196428666 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
30882024-08-28T02:18:05.973ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac0353ed-80eb-4c14-9eab-f26f965a394a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30892024-08-28T02:18:05.973ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac0353ed-80eb-4c14-9eab-f26f965a394a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30902024-08-28T02:18:05.975ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2874 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac0353ed-80eb-4c14-9eab-f26f965a394a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30912024-08-28T02:18:05.988ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1715 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = e3a94451-4bb1-4a39-88b5-91a1fa9955c1 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
30922024-08-28T02:18:06.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86949d7d-b51a-4448-8c95-1518d3f640b0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30932024-08-28T02:18:06.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86949d7d-b51a-4448-8c95-1518d3f640b0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30942024-08-28T02:18:06.999ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 86949d7d-b51a-4448-8c95-1518d3f640b0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
30952024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
30962024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
30972024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30982024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30992024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31002024-08-28T02:18:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31012024-08-28T02:18:07.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31022024-08-28T02:18:07.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31032024-08-28T02:18:07.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31042024-08-28T02:18:07.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.023099ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31052024-08-28T02:18:07.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31062024-08-28T02:18:07.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31072024-08-28T02:18:07.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31082024-08-28T02:18:07.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31092024-08-28T02:18:07.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31102024-08-28T02:18:07.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31112024-08-28T02:18:07.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
31122024-08-28T02:18:07.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31132024-08-28T02:18:07.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31142024-08-28T02:18:07.166ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
31152024-08-28T02:18:07.166ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.12914ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
31162024-08-28T02:18:08.018ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7912d5c9-0777-4118-aa63-ef1643ca95a5 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31172024-08-28T02:18:08.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7912d5c9-0777-4118-aa63-ef1643ca95a5 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31182024-08-28T02:18:08.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1875 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7912d5c9-0777-4118-aa63-ef1643ca95a5 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31192024-08-28T02:18:09.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5b234e2c-1b9c-47c2-a217-e32284de325e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31202024-08-28T02:18:09.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5b234e2c-1b9c-47c2-a217-e32284de325e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31212024-08-28T02:18:09.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3282 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5b234e2c-1b9c-47c2-a217-e32284de325e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31222024-08-28T02:18:10.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a49d85d4-19c5-4358-8274-6764eb65b6fc roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31232024-08-28T02:18:10.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a49d85d4-19c5-4358-8274-6764eb65b6fc resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31242024-08-28T02:18:10.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2342 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a49d85d4-19c5-4358-8274-6764eb65b6fc response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31252024-08-28T02:18:11.083ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df05e5a6-fc0e-4ba1-8651-4970d186fb78 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31262024-08-28T02:18:11.084ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df05e5a6-fc0e-4ba1-8651-4970d186fb78 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31272024-08-28T02:18:11.086ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3040 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = df05e5a6-fc0e-4ba1-8651-4970d186fb78 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31282024-08-28T02:18:12.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d6f3a06e-ee95-4dc2-95f6-7de7587cd742 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31292024-08-28T02:18:12.106ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d6f3a06e-ee95-4dc2-95f6-7de7587cd742 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31302024-08-28T02:18:12.108ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2259 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d6f3a06e-ee95-4dc2-95f6-7de7587cd742 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31312024-08-28T02:18:13.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54159758-276a-4e64-a05d-bba10372281f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31322024-08-28T02:18:13.128ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54159758-276a-4e64-a05d-bba10372281f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31332024-08-28T02:18:13.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2051 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54159758-276a-4e64-a05d-bba10372281f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31342024-08-28T02:18:14.149ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f79632a6-b2a9-4fd1-8dbf-614e6a071c96 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31352024-08-28T02:18:14.150ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f79632a6-b2a9-4fd1-8dbf-614e6a071c96 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31362024-08-28T02:18:14.151ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2696 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f79632a6-b2a9-4fd1-8dbf-614e6a071c96 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31372024-08-28T02:18:15.170ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e8e8ee11-e064-4488-86b7-41e292102ec0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31382024-08-28T02:18:15.171ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e8e8ee11-e064-4488-86b7-41e292102ec0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31392024-08-28T02:18:15.172ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2431 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e8e8ee11-e064-4488-86b7-41e292102ec0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31402024-08-28T02:18:15.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2104 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 21cf767c-9950-4bab-905d-51255509df43 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
31412024-08-28T02:18:16.192ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0a6f9eb-19ff-484e-b93e-d9519f21395a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31422024-08-28T02:18:16.193ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0a6f9eb-19ff-484e-b93e-d9519f21395a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31432024-08-28T02:18:16.195ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2505 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0a6f9eb-19ff-484e-b93e-d9519f21395a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
31442024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
31452024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
31462024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31472024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
31482024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
31492024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31502024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
31512024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
31522024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
31532024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31542024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31552024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
31562024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31572024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
31582024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
31592024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31602024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31612024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
31622024-08-28T02:18:17.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
31632024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31642024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
31652024-08-28T02:18:17.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
31662024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
31672024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
31682024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
31692024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
31702024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31712024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
31722024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31732024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
31742024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31752024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
31762024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
31772024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31782024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
31792024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
31802024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
31812024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31822024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
31832024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
31842024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31852024-08-28T02:18:17.145ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-28 02:08:17.145658867 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
31862024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
31872024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
31882024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
31892024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31902024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31912024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
31922024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
31932024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
31942024-08-28T02:18:17.145ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758
31952024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.66\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758"}
31962024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
31972024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
31982024-08-28T02:18:17.145ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
31992024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
32002024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
32012024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
32022024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32032024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32042024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
32052024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32062024-08-28T02:18:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32072024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
32082024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32092024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32102024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32112024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
32122024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32132024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32142024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32152024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32162024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32172024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32182024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32192024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
32202024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
32212024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
32222024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32232024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32242024-08-28T02:18:17.146ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6926ac43-4e2e-4378-9bce-42019ec72a30.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1305, tv_nsec: 304658633 } })
32252024-08-28T02:18:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32262024-08-28T02:18:17.147ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
32272024-08-28T02:18:17.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.50506ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
32282024-08-28T02:18:17.147ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("01868911-e8e4-4a77-b105-b83672ce13b2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c854ff-da18-4c2a-9d2f-7e15248c76a8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1305, tv_nsec: 305554071 } })
32292024-08-28T02:18:17.148ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32302024-08-28T02:18:17.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32312024-08-28T02:18:17.148ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
32322024-08-28T02:18:17.148ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.039911ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
32332024-08-28T02:18:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.661893ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
32342024-08-28T02:18:17.149ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32352024-08-28T02:18:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.216183ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32362024-08-28T02:18:17.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32372024-08-28T02:18:17.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1305, tv_nsec: 309524229 } })
32382024-08-28T02:18:17.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32392024-08-28T02:18:17.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32402024-08-28T02:18:17.151ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
32412024-08-28T02:18:17.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32422024-08-28T02:18:17.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32432024-08-28T02:18:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32442024-08-28T02:18:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32452024-08-28T02:18:17.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32462024-08-28T02:18:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32472024-08-28T02:18:17.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32482024-08-28T02:18:17.155ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32492024-08-28T02:18:17.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32502024-08-28T02:18:17.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32512024-08-28T02:18:17.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32522024-08-28T02:18:17.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32532024-08-28T02:18:17.158ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32542024-08-28T02:18:17.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
32552024-08-28T02:18:17.158ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32562024-08-28T02:18:17.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32572024-08-28T02:18:17.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32582024-08-28T02:18:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32592024-08-28T02:18:17.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32602024-08-28T02:18:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32612024-08-28T02:18:17.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32622024-08-28T02:18:17.161ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 16.100759ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
32632024-08-28T02:18:17.161ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32642024-08-28T02:18:17.162ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-28 02:08:17.145658867 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
32652024-08-28T02:18:17.162ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 16.458917ms iteration = 8 status = {"expiration":"2024-08-28T02:08:17.145658867Z","pruned":[]}
32662024-08-28T02:18:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32672024-08-28T02:18:17.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32682024-08-28T02:18:17.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32692024-08-28T02:18:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32702024-08-28T02:18:17.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
32712024-08-28T02:18:17.164ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
32722024-08-28T02:18:17.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
32732024-08-28T02:18:17.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1305, tv_nsec: 323270413 } })
32742024-08-28T02:18:17.165ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
32752024-08-28T02:18:17.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32762024-08-28T02:18:17.165ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32772024-08-28T02:18:17.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32782024-08-28T02:18:17.166ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 21.309343ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32792024-08-28T02:18:17.167ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32802024-08-28T02:18:17.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32812024-08-28T02:18:17.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32822024-08-28T02:18:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32832024-08-28T02:18:17.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32842024-08-28T02:18:17.169ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32852024-08-28T02:18:17.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32862024-08-28T02:18:17.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32872024-08-28T02:18:17.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32882024-08-28T02:18:17.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32892024-08-28T02:18:17.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32902024-08-28T02:18:17.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32912024-08-28T02:18:17.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32922024-08-28T02:18:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32932024-08-28T02:18:17.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942024-08-28T02:18:17.175ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32952024-08-28T02:18:17.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32962024-08-28T02:18:17.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1305, tv_nsec: 334015226 } })
32972024-08-28T02:18:17.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
32982024-08-28T02:18:17.176ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
32992024-08-28T02:18:17.176ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 31.072587ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
33002024-08-28T02:18:17.176ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33012024-08-28T02:18:17.177ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33022024-08-28T02:18:17.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33032024-08-28T02:18:17.178ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33042024-08-28T02:18:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33052024-08-28T02:18:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
33062024-08-28T02:18:17.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33072024-08-28T02:18:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33082024-08-28T02:18:17.180ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33092024-08-28T02:18:17.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33102024-08-28T02:18:17.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
33112024-08-28T02:18:17.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33122024-08-28T02:18:17.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33132024-08-28T02:18:17.182ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33142024-08-28T02:18:17.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33152024-08-28T02:18:17.185ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 40.514464ms iteration = 15 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
33162024-08-28T02:18:17.186ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 40.641687ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
33172024-08-28T02:18:17.187ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33182024-08-28T02:18:17.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33192024-08-28T02:18:17.187ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33202024-08-28T02:18:17.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33212024-08-28T02:18:17.191ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
33222024-08-28T02:18:17.191ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 45.625066ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
33232024-08-28T02:18:17.192ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
33242024-08-28T02:18:17.192ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
33252024-08-28T02:18:17.192ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 47.299489ms iteration = 8 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
33262024-08-28T02:18:17.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33272024-08-28T02:18:17.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33282024-08-28T02:18:17.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33292024-08-28T02:18:17.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33302024-08-28T02:18:17.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33312024-08-28T02:18:17.204ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33322024-08-28T02:18:17.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33332024-08-28T02:18:17.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33342024-08-28T02:18:17.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33352024-08-28T02:18:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33362024-08-28T02:18:17.205ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
33372024-08-28T02:18:17.205ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
33382024-08-28T02:18:17.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33392024-08-28T02:18:17.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33402024-08-28T02:18:17.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33412024-08-28T02:18:17.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33422024-08-28T02:18:17.207ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
33432024-08-28T02:18:17.207ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
33442024-08-28T02:18:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 62.051502ms iteration = 15 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
33452024-08-28T02:18:17.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33462024-08-28T02:18:17.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33472024-08-28T02:18:17.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33482024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
33492024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33502024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33512024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
33522024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
33532024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
33542024-08-28T02:18:17.209ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33552024-08-28T02:18:17.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33562024-08-28T02:18:17.211ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 65.610073ms iteration = 15 status = 0
33572024-08-28T02:18:17.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33582024-08-28T02:18:17.213ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0264922-ed3b-4a2c-b0c9-6fcecd6839cc roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
33592024-08-28T02:18:17.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33602024-08-28T02:18:17.213ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33612024-08-28T02:18:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33622024-08-28T02:18:17.214ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0264922-ed3b-4a2c-b0c9-6fcecd6839cc resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
33632024-08-28T02:18:17.215ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2067 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a0264922-ed3b-4a2c-b0c9-6fcecd6839cc response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
33642024-08-28T02:18:17.216ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
33652024-08-28T02:18:17.216ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
33662024-08-28T02:18:17.216ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
33672024-08-28T02:18:17.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33682024-08-28T02:18:17.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33692024-08-28T02:18:17.217ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
33702024-08-28T02:18:17.217ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
33712024-08-28T02:18:17.217ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 71.443268ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
33722024-08-28T02:18:17.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33732024-08-28T02:18:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33742024-08-28T02:18:17.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33752024-08-28T02:18:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33762024-08-28T02:18:17.224ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33772024-08-28T02:18:17.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33782024-08-28T02:18:17.231ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33792024-08-28T02:18:17.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33802024-08-28T02:18:17.233ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33812024-08-28T02:18:17.235ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33822024-08-28T02:18:17.235ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33832024-08-28T02:18:17.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33842024-08-28T02:18:17.242ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-28T02:11:15.536604Z, time_modified: 2024-08-28T02:11:15.536604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
33852024-08-28T02:18:17.242ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 96.738752ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]}},"default_endpoint":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
33862024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
33872024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
33882024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33892024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33902024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
33912024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 0 version = 2
33922024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33932024-08-28T02:18:17.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33942024-08-28T02:18:17.256ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
33952024-08-28T02:18:17.256ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 111.484344ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
33962024-08-28T02:18:17.267ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
33972024-08-28T02:18:17.270ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
33982024-08-28T02:18:17.275ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 130.356239ms iteration = 18 status = {"physical_disks_added":0}
33992024-08-28T02:18:17.279ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
34002024-08-28T02:18:17.279ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34012024-08-28T02:18:17.279ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c4f9c1e-854f-4b04-8f67-bde172051401", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34022024-08-28T02:18:17.279ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34032024-08-28T02:18:17.281ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
34042024-08-28T02:18:17.317ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
34052024-08-28T02:18:17.317ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34062024-08-28T02:18:17.317ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3ea308ff-295a-448e-89ff-d13874146d86", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34072024-08-28T02:18:17.317ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34082024-08-28T02:18:17.319ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
34092024-08-28T02:18:17.335ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
34102024-08-28T02:18:17.335ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
34112024-08-28T02:18:17.336ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "948e1989-5ad9-4bd1-8ef4-5aa6ae85ff4f", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34122024-08-28T02:18:17.336ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
34132024-08-28T02:18:17.338ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
34142024-08-28T02:18:17.433ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34152024-08-28T02:18:17.434ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "338ec564-7f51-43c7-afbf-8bcd1b31d603", "content-length": "372", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} })
34162024-08-28T02:18:17.434ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34172024-08-28T02:18:17.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34182024-08-28T02:18:17.435ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34192024-08-28T02:18:17.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34202024-08-28T02:18:17.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34212024-08-28T02:18:17.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34222024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34232024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34242024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34252024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34262024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34272024-08-28T02:18:17.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34282024-08-28T02:18:17.444ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
34292024-08-28T02:18:17.467ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
34302024-08-28T02:18:17.468ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e9999112-dd5c-42fe-83bf-b51d177c7d19", "content-length": "691", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} })
34312024-08-28T02:18:17.468ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
34322024-08-28T02:18:17.468ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
34332024-08-28T02:18:17.468ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
34342024-08-28T02:18:17.469ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 323.718928ms iteration = 16 status = {}
34352024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34362024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34372024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 1 version = 2
34382024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
34392024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-28 02:11:35.148647 UTC dns_group = external generation = 2
34402024-08-28T02:18:17.470ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 325.093366ms iteration = 9 status = {"generation":2}
34412024-08-28T02:18:17.473ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34422024-08-28T02:18:17.473ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
34432024-08-28T02:18:17.473ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
34442024-08-28T02:18:17.473ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-28 02:11:34.295733 UTC dns_group = internal generation = 1
34452024-08-28T02:18:17.473ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 327.716177ms iteration = 9 status = {"generation":1}
34462024-08-28T02:18:17.474ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "88b49787-2822-42f2-8c36-d6464caf3505", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} })
34472024-08-28T02:18:17.475ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 329.787498ms iteration = 17 status = {}
34482024-08-28T02:18:17.481ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 335.598135ms iteration = 15 status = {}
34492024-08-28T02:18:17.487ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
34502024-08-28T02:18:17.487ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a0aa65b6-67db-469f-affa-3108be94e62b", "content-length": "2", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34512024-08-28T02:18:17.489ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34522024-08-28T02:18:17.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34532024-08-28T02:18:17.491ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 345.782966ms iteration = 15 status = null
34542024-08-28T02:18:17.529ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
34552024-08-28T02:18:17.530ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b2a7116e-296d-42b5-a563-0017fc8e2bd5", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34562024-08-28T02:18:17.530ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34572024-08-28T02:18:17.532ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "013ead30-c061-49f7-9a46-5572452ab663", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34582024-08-28T02:18:17.532ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34592024-08-28T02:18:17.534ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
34602024-08-28T02:18:17.534ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 389.046975ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
34612024-08-28T02:18:17.548ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
34622024-08-28T02:18:17.548ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "be69818b-c5b8-4675-9733-d04b0a284a43", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34632024-08-28T02:18:17.548ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34642024-08-28T02:18:17.550ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9292f815-f3ec-4250-8ed8-720106fc6bc6", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34652024-08-28T02:18:17.550ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34662024-08-28T02:18:17.551ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e1223e57-3f95-4c6f-ab15-8ecf458a879f", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34672024-08-28T02:18:17.551ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34682024-08-28T02:18:17.552ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
34692024-08-28T02:18:17.552ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 407.38494ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
34702024-08-28T02:18:17.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34712024-08-28T02:18:17.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34722024-08-28T02:18:17.615ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
34732024-08-28T02:18:17.615ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34742024-08-28T02:18:17.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34752024-08-28T02:18:17.624ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
34762024-08-28T02:18:17.624ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a189e773-c7c4-4971-9060-1cac30604d44", "content-length": "121", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34772024-08-28T02:18:17.624ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
34782024-08-28T02:18:17.624ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
34792024-08-28T02:18:17.625ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
34802024-08-28T02:18:17.625ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
34812024-08-28T02:18:17.625ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
34822024-08-28T02:18:17.625ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
34832024-08-28T02:18:17.625ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
34842024-08-28T02:18:17.625ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34852024-08-28T02:18:17.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34862024-08-28T02:18:17.627ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
34872024-08-28T02:18:17.627ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ffd760ed-e92b-4877-9399-91a6f5419bff", "content-length": "2", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34882024-08-28T02:18:17.627ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
34892024-08-28T02:18:17.627ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7130af91-d909-4742-9334-8a999feeefbc", "content-length": "51", "date": "Wed, 28 Aug 2024 02:18:16 GMT"} })
34902024-08-28T02:18:17.627ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
34912024-08-28T02:18:17.627ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34922024-08-28T02:18:17.720ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "232b6151-cf85-471b-b7f8-8d052ed0464b", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} })
34932024-08-28T02:18:17.720ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
34942024-08-28T02:18:17.720ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34952024-08-28T02:18:17.720ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ecd23803-d8bf-4237-b4eb-d2d826dd1d16", "content-length": "667", "date": "Wed, 28 Aug 2024 02:18:17 GMT"} })
34962024-08-28T02:18:17.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34972024-08-28T02:18:17.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34982024-08-28T02:18:17.724ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34992024-08-28T02:18:17.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35002024-08-28T02:18:17.727ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35012024-08-28T02:18:17.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35022024-08-28T02:18:17.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35032024-08-28T02:18:17.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35042024-08-28T02:18:17.737ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 591.427994ms iteration = 15 status = {}
35052024-08-28T02:18:18.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27a820ba-b41e-4450-a788-2718669f6337 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35062024-08-28T02:18:18.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27a820ba-b41e-4450-a788-2718669f6337 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35072024-08-28T02:18:18.239ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2382 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 27a820ba-b41e-4450-a788-2718669f6337 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35082024-08-28T02:18:19.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7f6f310d-4cb5-41e8-8274-8eb1a61f2091 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35092024-08-28T02:18:19.259ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7f6f310d-4cb5-41e8-8274-8eb1a61f2091 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35102024-08-28T02:18:19.261ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2476 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7f6f310d-4cb5-41e8-8274-8eb1a61f2091 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35112024-08-28T02:18:20.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7e43621-ac09-4d60-8964-7d94949d7414 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35122024-08-28T02:18:20.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7e43621-ac09-4d60-8964-7d94949d7414 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35132024-08-28T02:18:20.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2923 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c7e43621-ac09-4d60-8964-7d94949d7414 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35142024-08-28T02:18:21.302ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9e1b8720-aba1-48ba-b0ac-b3bdb9e118d1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35152024-08-28T02:18:21.302ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9e1b8720-aba1-48ba-b0ac-b3bdb9e118d1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35162024-08-28T02:18:21.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2603 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9e1b8720-aba1-48ba-b0ac-b3bdb9e118d1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35172024-08-28T02:18:22.323ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee2a2711-8ee6-494d-91ee-e014bdcc8ed9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35182024-08-28T02:18:22.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee2a2711-8ee6-494d-91ee-e014bdcc8ed9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35192024-08-28T02:18:22.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3097 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ee2a2711-8ee6-494d-91ee-e014bdcc8ed9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35202024-08-28T02:18:23.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f68d8646-c5d2-46f0-af06-ad0fa73c8fed roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35212024-08-28T02:18:23.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f68d8646-c5d2-46f0-af06-ad0fa73c8fed resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35222024-08-28T02:18:23.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2694 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f68d8646-c5d2-46f0-af06-ad0fa73c8fed response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35232024-08-28T02:18:24.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0197776-56b0-448d-962c-2ea5780a7fca roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35242024-08-28T02:18:24.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0197776-56b0-448d-962c-2ea5780a7fca resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35252024-08-28T02:18:24.368ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2214 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e0197776-56b0-448d-962c-2ea5780a7fca response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35262024-08-28T02:18:25.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11843dee-42a2-41c5-b619-3ed8158154b1 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35272024-08-28T02:18:25.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11843dee-42a2-41c5-b619-3ed8158154b1 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35282024-08-28T02:18:25.388ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1915 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11843dee-42a2-41c5-b619-3ed8158154b1 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35292024-08-28T02:18:25.990ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2534 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 44b0a77a-19cd-417d-99a5-da86a47ec395 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
35302024-08-28T02:18:26.407ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9803697c-d2fa-4c00-b19d-2fb0d6f5543f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35312024-08-28T02:18:26.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9803697c-d2fa-4c00-b19d-2fb0d6f5543f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35322024-08-28T02:18:26.409ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2005 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9803697c-d2fa-4c00-b19d-2fb0d6f5543f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35332024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
35342024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35352024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
35362024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35372024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35382024-08-28T02:18:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35392024-08-28T02:18:27.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35402024-08-28T02:18:27.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35412024-08-28T02:18:27.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35422024-08-28T02:18:27.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35432024-08-28T02:18:27.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35442024-08-28T02:18:27.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35452024-08-28T02:18:27.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35462024-08-28T02:18:27.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
35472024-08-28T02:18:27.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35482024-08-28T02:18:27.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35492024-08-28T02:18:27.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
35502024-08-28T02:18:27.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.125629ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
35512024-08-28T02:18:27.179ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35522024-08-28T02:18:27.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35532024-08-28T02:18:27.205ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.42734ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35542024-08-28T02:18:27.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 52a3e551-4e13-40bb-a74a-e06dc4142d8f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35552024-08-28T02:18:27.429ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 52a3e551-4e13-40bb-a74a-e06dc4142d8f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35562024-08-28T02:18:27.431ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2756 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 52a3e551-4e13-40bb-a74a-e06dc4142d8f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35572024-08-28T02:18:28.450ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1a34d83-9546-412e-a456-e56fb3e105a7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35582024-08-28T02:18:28.450ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1a34d83-9546-412e-a456-e56fb3e105a7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35592024-08-28T02:18:28.452ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2507 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1a34d83-9546-412e-a456-e56fb3e105a7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35602024-08-28T02:18:29.472ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51cf2f2f-f27d-4e30-a702-7a4d196eb59a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35612024-08-28T02:18:29.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51cf2f2f-f27d-4e30-a702-7a4d196eb59a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35622024-08-28T02:18:29.474ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1935 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51cf2f2f-f27d-4e30-a702-7a4d196eb59a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35632024-08-28T02:18:30.493ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5153dc67-e1c8-46af-83cf-6e5885877c0f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35642024-08-28T02:18:30.493ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5153dc67-e1c8-46af-83cf-6e5885877c0f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35652024-08-28T02:18:30.495ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5153dc67-e1c8-46af-83cf-6e5885877c0f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35662024-08-28T02:18:31.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0173b380-c467-4829-bade-43498bf7d1b8 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35672024-08-28T02:18:31.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0173b380-c467-4829-bade-43498bf7d1b8 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35682024-08-28T02:18:31.517ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2333 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0173b380-c467-4829-bade-43498bf7d1b8 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35692024-08-28T02:18:32.535ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6594c3b9-08ed-4197-8243-517d051128c7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35702024-08-28T02:18:32.536ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6594c3b9-08ed-4197-8243-517d051128c7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35712024-08-28T02:18:32.538ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2706 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6594c3b9-08ed-4197-8243-517d051128c7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35722024-08-28T02:18:33.557ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ce403aa-48c0-46aa-bc7a-59b630699b47 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35732024-08-28T02:18:33.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ce403aa-48c0-46aa-bc7a-59b630699b47 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35742024-08-28T02:18:33.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2611 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ce403aa-48c0-46aa-bc7a-59b630699b47 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35752024-08-28T02:18:34.578ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c919c3-5149-44ce-98ce-19ea4b5e8c5d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35762024-08-28T02:18:34.578ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c919c3-5149-44ce-98ce-19ea4b5e8c5d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35772024-08-28T02:18:34.580ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1908 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 37c919c3-5149-44ce-98ce-19ea4b5e8c5d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35782024-08-28T02:18:35.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 273f0f51-bc07-4f4f-92bf-a467a8cbf503 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35792024-08-28T02:18:35.600ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 273f0f51-bc07-4f4f-92bf-a467a8cbf503 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35802024-08-28T02:18:35.602ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2679 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 273f0f51-bc07-4f4f-92bf-a467a8cbf503 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35812024-08-28T02:18:35.743ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7803fae5-74cd-483a-a7cc-f2fb0907f5f2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
35822024-08-28T02:18:35.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7803fae5-74cd-483a-a7cc-f2fb0907f5f2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
35832024-08-28T02:18:35.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3508 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 7803fae5-74cd-483a-a7cc-f2fb0907f5f2 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
35842024-08-28T02:18:35.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1729 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = bcb1c0b7-efd7-40e3-a856-c0a026553f15 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
35852024-08-28T02:18:36.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82162a61-d056-4c53-95cc-37f55fd1c259 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35862024-08-28T02:18:36.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82162a61-d056-4c53-95cc-37f55fd1c259 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35872024-08-28T02:18:36.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2450 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 82162a61-d056-4c53-95cc-37f55fd1c259 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
35882024-08-28T02:18:37.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
35892024-08-28T02:18:37.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
35902024-08-28T02:18:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35912024-08-28T02:18:37.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35922024-08-28T02:18:37.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35932024-08-28T02:18:37.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35942024-08-28T02:18:37.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35952024-08-28T02:18:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35962024-08-28T02:18:37.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35972024-08-28T02:18:37.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35982024-08-28T02:18:37.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35992024-08-28T02:18:37.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36002024-08-28T02:18:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36012024-08-28T02:18:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
36022024-08-28T02:18:37.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36032024-08-28T02:18:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36042024-08-28T02:18:37.164ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
36052024-08-28T02:18:37.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.747942ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
36062024-08-28T02:18:37.174ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36072024-08-28T02:18:37.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36082024-08-28T02:18:37.203ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.888289ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36092024-08-28T02:18:37.642ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a535a0f-670e-4e5c-83fd-7aa4999c4f4a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36102024-08-28T02:18:37.642ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a535a0f-670e-4e5c-83fd-7aa4999c4f4a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36112024-08-28T02:18:37.643ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1826 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2a535a0f-670e-4e5c-83fd-7aa4999c4f4a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36122024-08-28T02:18:38.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7563b28c-c69a-4a55-8708-418a986b9775 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36132024-08-28T02:18:38.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7563b28c-c69a-4a55-8708-418a986b9775 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36142024-08-28T02:18:38.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2943 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7563b28c-c69a-4a55-8708-418a986b9775 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36152024-08-28T02:18:39.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a859c95f-36b5-4cff-82d7-fe016531bc11 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36162024-08-28T02:18:39.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a859c95f-36b5-4cff-82d7-fe016531bc11 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36172024-08-28T02:18:39.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2377 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a859c95f-36b5-4cff-82d7-fe016531bc11 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36182024-08-28T02:18:40.705ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 310244e7-d1be-4fd7-a28e-737063bf4466 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36192024-08-28T02:18:40.706ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 310244e7-d1be-4fd7-a28e-737063bf4466 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36202024-08-28T02:18:40.708ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2822 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 310244e7-d1be-4fd7-a28e-737063bf4466 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36212024-08-28T02:18:41.727ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1edd1469-ed11-482d-abb6-3e843df4ab06 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36222024-08-28T02:18:41.727ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1edd1469-ed11-482d-abb6-3e843df4ab06 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36232024-08-28T02:18:41.730ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2639 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1edd1469-ed11-482d-abb6-3e843df4ab06 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36242024-08-28T02:18:42.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 956acfe1-57c4-488e-a9f2-2fc8f617d033 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36252024-08-28T02:18:42.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 956acfe1-57c4-488e-a9f2-2fc8f617d033 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36262024-08-28T02:18:42.751ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1974 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 956acfe1-57c4-488e-a9f2-2fc8f617d033 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36272024-08-28T02:18:43.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a593357f-3a47-4ed5-8afe-52448383f6ce roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36282024-08-28T02:18:43.772ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a593357f-3a47-4ed5-8afe-52448383f6ce resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36292024-08-28T02:18:43.774ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3068 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a593357f-3a47-4ed5-8afe-52448383f6ce response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36302024-08-28T02:18:44.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 31fa0c41-a2db-4998-b738-730ac429f00c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36312024-08-28T02:18:44.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 31fa0c41-a2db-4998-b738-730ac429f00c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36322024-08-28T02:18:44.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2556 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 31fa0c41-a2db-4998-b738-730ac429f00c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36332024-08-28T02:18:45.816ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7075fc0c-5c42-4662-a887-e500a1800264 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36342024-08-28T02:18:45.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7075fc0c-5c42-4662-a887-e500a1800264 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36352024-08-28T02:18:45.818ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2105 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7075fc0c-5c42-4662-a887-e500a1800264 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36362024-08-28T02:18:45.988ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1642 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = e8738d9c-2ad2-4eaa-880c-ef6f076cb3ec response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
36372024-08-28T02:18:46.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51fe1d84-286a-4917-a3ff-41dd2d5e57e9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36382024-08-28T02:18:46.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51fe1d84-286a-4917-a3ff-41dd2d5e57e9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36392024-08-28T02:18:46.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2297 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 51fe1d84-286a-4917-a3ff-41dd2d5e57e9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
36402024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
36412024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
36422024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
36432024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36442024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
36452024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36462024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
36472024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
36482024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
36492024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36502024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36512024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36522024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
36532024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36542024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
36552024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
36562024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36572024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
36582024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36592024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
36602024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
36612024-08-28T02:18:47.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
36622024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36632024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
36642024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
36652024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36662024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36672024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
36682024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
36692024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36702024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36712024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
36722024-08-28T02:18:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36732024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36742024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36752024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36762024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36772024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36782024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36792024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36802024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36812024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36822024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36832024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36842024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36852024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36862024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36872024-08-28T02:18:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36882024-08-28T02:18:47.148ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.962899ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36892024-08-28T02:18:47.149ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1335, tv_nsec: 307727594 } })
36902024-08-28T02:18:47.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
36912024-08-28T02:18:47.151ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36922024-08-28T02:18:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36932024-08-28T02:18:47.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1335, tv_nsec: 309936288 } })
36942024-08-28T02:18:47.152ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
36952024-08-28T02:18:47.152ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36962024-08-28T02:18:47.153ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36972024-08-28T02:18:47.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36982024-08-28T02:18:47.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
36992024-08-28T02:18:47.156ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37002024-08-28T02:18:47.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37012024-08-28T02:18:47.156ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37022024-08-28T02:18:47.156ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37032024-08-28T02:18:47.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37042024-08-28T02:18:47.157ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37052024-08-28T02:18:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37062024-08-28T02:18:47.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37072024-08-28T02:18:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37082024-08-28T02:18:47.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37092024-08-28T02:18:47.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37102024-08-28T02:18:47.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37112024-08-28T02:18:47.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37122024-08-28T02:18:47.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
37132024-08-28T02:18:47.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37142024-08-28T02:18:47.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37152024-08-28T02:18:47.176ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
37162024-08-28T02:18:47.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.18419ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
37172024-08-28T02:18:47.178ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
37182024-08-28T02:18:47.178ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
37192024-08-28T02:18:47.178ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37202024-08-28T02:18:47.178ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37212024-08-28T02:18:47.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37222024-08-28T02:18:47.181ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
37232024-08-28T02:18:47.181ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
37242024-08-28T02:18:47.181ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 35.395992ms iteration = 16 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
37252024-08-28T02:18:47.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37262024-08-28T02:18:47.181ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37272024-08-28T02:18:47.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37282024-08-28T02:18:47.182ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 36.295691ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
37292024-08-28T02:18:47.182ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37302024-08-28T02:18:47.182ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37312024-08-28T02:18:47.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37322024-08-28T02:18:47.183ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37332024-08-28T02:18:47.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37342024-08-28T02:18:47.183ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37352024-08-28T02:18:47.183ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37362024-08-28T02:18:47.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37372024-08-28T02:18:47.184ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37382024-08-28T02:18:47.184ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 38.981054ms iteration = 16 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
37392024-08-28T02:18:47.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
37402024-08-28T02:18:47.184ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37412024-08-28T02:18:47.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37422024-08-28T02:18:47.185ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 39.415112ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
37432024-08-28T02:18:47.185ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37442024-08-28T02:18:47.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37452024-08-28T02:18:47.186ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
37462024-08-28T02:18:47.186ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37472024-08-28T02:18:47.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37482024-08-28T02:18:47.187ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37492024-08-28T02:18:47.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37502024-08-28T02:18:47.187ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37512024-08-28T02:18:47.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37522024-08-28T02:18:47.187ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1335, tv_nsec: 345725188 } })
37532024-08-28T02:18:47.188ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
37542024-08-28T02:18:47.188ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37552024-08-28T02:18:47.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37562024-08-28T02:18:47.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37572024-08-28T02:18:47.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37582024-08-28T02:18:47.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37592024-08-28T02:18:47.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37602024-08-28T02:18:47.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37612024-08-28T02:18:47.190ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
37622024-08-28T02:18:47.190ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 44.527175ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
37632024-08-28T02:18:47.192ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 46.912032ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37642024-08-28T02:18:47.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37652024-08-28T02:18:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37662024-08-28T02:18:47.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37672024-08-28T02:18:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37682024-08-28T02:18:47.223ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 77.874446ms iteration = 19 status = {"physical_disks_added":0}
37692024-08-28T02:18:47.231ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
37702024-08-28T02:18:47.231ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.337864ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
37712024-08-28T02:18:47.236ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
37722024-08-28T02:18:47.236ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37732024-08-28T02:18:47.237ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9243c30c-59bb-448d-8ea9-33901ba2f29b", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:47 GMT"} })
37742024-08-28T02:18:47.237ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37752024-08-28T02:18:47.239ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37762024-08-28T02:18:47.240ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
37772024-08-28T02:18:47.240ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37782024-08-28T02:18:47.240ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b51b65fa-ca43-41bd-bb51-23e5ce4d71e8", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:47 GMT"} })
37792024-08-28T02:18:47.240ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37802024-08-28T02:18:47.242ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37812024-08-28T02:18:47.254ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37822024-08-28T02:18:47.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37832024-08-28T02:18:47.293ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
37842024-08-28T02:18:47.293ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37852024-08-28T02:18:47.294ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9445874b-73e2-4e4d-93f1-de61df721aad", "content-length": "26", "date": "Wed, 28 Aug 2024 02:18:47 GMT"} })
37862024-08-28T02:18:47.294ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37872024-08-28T02:18:47.295ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
37882024-08-28T02:18:47.296ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
37892024-08-28T02:18:47.296ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "296f2f3b-139c-4824-b701-b4115dc85bfb", "content-length": "372", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
37902024-08-28T02:18:47.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37912024-08-28T02:18:47.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37922024-08-28T02:18:47.298ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37932024-08-28T02:18:47.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37942024-08-28T02:18:47.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37952024-08-28T02:18:47.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37962024-08-28T02:18:47.300ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37972024-08-28T02:18:47.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37982024-08-28T02:18:47.302ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37992024-08-28T02:18:47.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38002024-08-28T02:18:47.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38012024-08-28T02:18:47.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38022024-08-28T02:18:47.304ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38032024-08-28T02:18:47.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38042024-08-28T02:18:47.309ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38052024-08-28T02:18:47.310ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "372b96c4-59d7-4289-a90a-e63771311fb9", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38062024-08-28T02:18:47.312ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 166.270511ms iteration = 18 status = {}
38072024-08-28T02:18:47.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38082024-08-28T02:18:47.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
38092024-08-28T02:18:47.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38102024-08-28T02:18:47.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38112024-08-28T02:18:47.332ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 186.589285ms iteration = 16 status = 0
38122024-08-28T02:18:47.339ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
38132024-08-28T02:18:47.339ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5f56c669-3966-4416-85c3-e7f17960c4a0", "content-length": "2", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38142024-08-28T02:18:47.341ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38152024-08-28T02:18:47.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38162024-08-28T02:18:47.342ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 197.152575ms iteration = 16 status = null
38172024-08-28T02:18:47.381ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 235.494486ms iteration = 16 status = {}
38182024-08-28T02:18:47.388ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
38192024-08-28T02:18:47.389ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "59ca445e-96e7-4484-a86d-1684f9d44879", "content-length": "691", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38202024-08-28T02:18:47.389ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
38212024-08-28T02:18:47.389ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
38222024-08-28T02:18:47.389ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
38232024-08-28T02:18:47.390ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 244.386133ms iteration = 17 status = {}
38242024-08-28T02:18:47.459ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38252024-08-28T02:18:47.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38262024-08-28T02:18:47.462ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
38272024-08-28T02:18:47.462ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38282024-08-28T02:18:47.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38292024-08-28T02:18:47.469ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
38302024-08-28T02:18:47.469ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "51551e41-1962-4847-afa6-ebac037d07e9", "content-length": "121", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38312024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
38322024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
38332024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
38342024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
38352024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
38362024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
38372024-08-28T02:18:47.469ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
38382024-08-28T02:18:47.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38392024-08-28T02:18:47.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38402024-08-28T02:18:47.471ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
38412024-08-28T02:18:47.471ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a945b479-2fb8-4960-89a3-242544b54e9a", "content-length": "2", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38422024-08-28T02:18:47.471ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
38432024-08-28T02:18:47.471ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "73b56ee8-52c2-4755-b3af-fec694f50337", "content-length": "51", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38442024-08-28T02:18:47.471ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
38452024-08-28T02:18:47.471ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
38462024-08-28T02:18:47.560ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "892f39b9-5600-416e-952c-7bc2684632ae", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38472024-08-28T02:18:47.560ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
38482024-08-28T02:18:47.560ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
38492024-08-28T02:18:47.560ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1a4927a0-cc8e-4d46-8fed-533b0a23ae37", "content-length": "667", "date": "Wed, 28 Aug 2024 02:18:46 GMT"} })
38502024-08-28T02:18:47.560ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38512024-08-28T02:18:47.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38522024-08-28T02:18:47.564ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38532024-08-28T02:18:47.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38542024-08-28T02:18:47.566ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38552024-08-28T02:18:47.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38562024-08-28T02:18:47.568ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38572024-08-28T02:18:47.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38582024-08-28T02:18:47.576ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 430.825935ms iteration = 16 status = {}
38592024-08-28T02:18:47.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a7a5f3c5-8586-4e67-bfac-0723c01f0aff roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38602024-08-28T02:18:47.856ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a7a5f3c5-8586-4e67-bfac-0723c01f0aff resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38612024-08-28T02:18:47.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2253 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a7a5f3c5-8586-4e67-bfac-0723c01f0aff response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38622024-08-28T02:18:48.877ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7296aa45-c5c2-4f38-abd9-3b01a3891f06 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38632024-08-28T02:18:48.877ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7296aa45-c5c2-4f38-abd9-3b01a3891f06 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38642024-08-28T02:18:48.879ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2331 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7296aa45-c5c2-4f38-abd9-3b01a3891f06 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38652024-08-28T02:18:49.898ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90b6b297-31a8-4c2f-9572-e1d3cf817d6a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38662024-08-28T02:18:49.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90b6b297-31a8-4c2f-9572-e1d3cf817d6a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38672024-08-28T02:18:49.900ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2058 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90b6b297-31a8-4c2f-9572-e1d3cf817d6a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38682024-08-28T02:18:50.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d34b9fc1-ef2b-428e-b5bc-0ebfeb72d75e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38692024-08-28T02:18:50.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d34b9fc1-ef2b-428e-b5bc-0ebfeb72d75e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38702024-08-28T02:18:50.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2316 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d34b9fc1-ef2b-428e-b5bc-0ebfeb72d75e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38712024-08-28T02:18:51.304ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:56502
38722024-08-28T02:18:51.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56502 req_id = 4f3cdfab-d667-4c8b-a6dd-2c86c56a3e1e roles = RoleSet { roles: {} } uri = /metrics/producers
38732024-08-28T02:18:51.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56502 req_id = 4f3cdfab-d667-4c8b-a6dd-2c86c56a3e1e resource = Database result = Ok(()) uri = /metrics/producers
38742024-08-28T02:18:51.340ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
38752024-08-28T02:18:51.340ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56502 req_id = 4f3cdfab-d667-4c8b-a6dd-2c86c56a3e1e roles = RoleSet { roles: {} } uri = /metrics/producers
38762024-08-28T02:18:51.340ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56502 req_id = 4f3cdfab-d667-4c8b-a6dd-2c86c56a3e1e resource = Database result = Ok(()) uri = /metrics/producers
38772024-08-28T02:18:51.346ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
38782024-08-28T02:18:51.346ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "21a36932-a601-41b5-8b08-e7ab48b3f009", "date": "Wed, 28 Aug 2024 02:18:51 GMT"} })
38792024-08-28T02:18:51.346ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = d65da7b7-eebe-4dd3-87b9-b0fa344c777a
38802024-08-28T02:18:51.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 43291 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:56502 req_id = 4f3cdfab-d667-4c8b-a6dd-2c86c56a3e1e response_code = 201 uri = /metrics/producers
38812024-08-28T02:18:51.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 20e4299f-7315-4925-bb82-9b9ec711f77c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38822024-08-28T02:18:51.938ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 20e4299f-7315-4925-bb82-9b9ec711f77c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38832024-08-28T02:18:51.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2251 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 20e4299f-7315-4925-bb82-9b9ec711f77c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38842024-08-28T02:18:52.958ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b61439b5-b9d7-4d30-806f-79895d6288ec roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38852024-08-28T02:18:52.959ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b61439b5-b9d7-4d30-806f-79895d6288ec resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38862024-08-28T02:18:52.961ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2394 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b61439b5-b9d7-4d30-806f-79895d6288ec response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38872024-08-28T02:18:53.552ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:54944
38882024-08-28T02:18:53.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54944 req_id = 91615573-5b0e-413f-be7b-ab88110f24d9 roles = RoleSet { roles: {} } uri = /metrics/producers
38892024-08-28T02:18:53.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54944 req_id = 91615573-5b0e-413f-be7b-ab88110f24d9 resource = Database result = Ok(()) uri = /metrics/producers
38902024-08-28T02:18:53.586ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
38912024-08-28T02:18:53.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54944 req_id = 91615573-5b0e-413f-be7b-ab88110f24d9 roles = RoleSet { roles: {} } uri = /metrics/producers
38922024-08-28T02:18:53.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54944 req_id = 91615573-5b0e-413f-be7b-ab88110f24d9 resource = Database result = Ok(()) uri = /metrics/producers
38932024-08-28T02:18:53.621ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
38942024-08-28T02:18:53.621ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3608de41-e1aa-4111-afec-e4d1b461e90d", "date": "Wed, 28 Aug 2024 02:18:53 GMT"} })
38952024-08-28T02:18:53.621ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
38962024-08-28T02:18:53.622ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 70445 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:54944 req_id = 91615573-5b0e-413f-be7b-ab88110f24d9 response_code = 201 uri = /metrics/producers
38972024-08-28T02:18:53.980ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa6c3206-07ef-44e0-81a4-d713539b1325 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38982024-08-28T02:18:53.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa6c3206-07ef-44e0-81a4-d713539b1325 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
38992024-08-28T02:18:53.982ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2408 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = aa6c3206-07ef-44e0-81a4-d713539b1325 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39002024-08-28T02:18:55.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9d3a16c6-32e1-4348-8bae-13997e1874ef roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39012024-08-28T02:18:55.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9d3a16c6-32e1-4348-8bae-13997e1874ef resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39022024-08-28T02:18:55.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9d3a16c6-32e1-4348-8bae-13997e1874ef response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39032024-08-28T02:18:55.990ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2560 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 88bafc26-d35b-431c-9149-70b8b90e1d4f response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
39042024-08-28T02:18:56.023ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25e320be-087e-48a5-947e-c9383dbe2791 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39052024-08-28T02:18:56.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25e320be-087e-48a5-947e-c9383dbe2791 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39062024-08-28T02:18:56.025ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1881 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 25e320be-087e-48a5-947e-c9383dbe2791 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39072024-08-28T02:18:57.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e06fcca9-6d6a-4fb9-96df-555882758246 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39082024-08-28T02:18:57.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e06fcca9-6d6a-4fb9-96df-555882758246 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39092024-08-28T02:18:57.047ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2172 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e06fcca9-6d6a-4fb9-96df-555882758246 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39102024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
39112024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39122024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
39132024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39142024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39152024-08-28T02:18:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39162024-08-28T02:18:57.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39172024-08-28T02:18:57.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39182024-08-28T02:18:57.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39192024-08-28T02:18:57.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.05346ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39202024-08-28T02:18:57.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39212024-08-28T02:18:57.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39222024-08-28T02:18:57.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39232024-08-28T02:18:57.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39242024-08-28T02:18:57.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39252024-08-28T02:18:57.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39262024-08-28T02:18:57.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
39272024-08-28T02:18:57.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39282024-08-28T02:18:57.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39292024-08-28T02:18:57.163ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
39302024-08-28T02:18:57.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.051248ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
39312024-08-28T02:18:58.067ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = cc24f178-a734-4502-8ba3-3635e4f0a9bc roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39322024-08-28T02:18:58.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = cc24f178-a734-4502-8ba3-3635e4f0a9bc resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39332024-08-28T02:18:58.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2074 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = cc24f178-a734-4502-8ba3-3635e4f0a9bc response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39342024-08-28T02:18:59.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7550cb81-2316-488d-8204-d5ed3e49be82 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39352024-08-28T02:18:59.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7550cb81-2316-488d-8204-d5ed3e49be82 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39362024-08-28T02:18:59.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2396 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7550cb81-2316-488d-8204-d5ed3e49be82 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39372024-08-28T02:19:00.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 514972f1-fbbc-4a4d-b482-0220d93db86b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39382024-08-28T02:19:00.112ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 514972f1-fbbc-4a4d-b482-0220d93db86b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39392024-08-28T02:19:00.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2697 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 514972f1-fbbc-4a4d-b482-0220d93db86b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39402024-08-28T02:19:01.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11a0d6f4-6b83-4e7b-9152-dcb9fa1c5799 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39412024-08-28T02:19:01.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11a0d6f4-6b83-4e7b-9152-dcb9fa1c5799 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39422024-08-28T02:19:01.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2625 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 11a0d6f4-6b83-4e7b-9152-dcb9fa1c5799 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39432024-08-28T02:19:02.156ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 68839ee1-671b-4512-baf6-4149a4529bd5 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39442024-08-28T02:19:02.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 68839ee1-671b-4512-baf6-4149a4529bd5 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39452024-08-28T02:19:02.158ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2397 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 68839ee1-671b-4512-baf6-4149a4529bd5 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39462024-08-28T02:19:03.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9f14f5b-b580-444a-a974-5be4017abbcb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39472024-08-28T02:19:03.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9f14f5b-b580-444a-a974-5be4017abbcb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39482024-08-28T02:19:03.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2431 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9f14f5b-b580-444a-a974-5be4017abbcb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39492024-08-28T02:19:03.667ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64134
39502024-08-28T02:19:03.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64134 req_id = 46623a08-8755-4840-b54a-5f5feb2fba72 roles = RoleSet { roles: {} } uri = /metrics/producers
39512024-08-28T02:19:03.668ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64134 req_id = 46623a08-8755-4840-b54a-5f5feb2fba72 resource = Database result = Ok(()) uri = /metrics/producers
39522024-08-28T02:19:03.701ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
39532024-08-28T02:19:03.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64134 req_id = 46623a08-8755-4840-b54a-5f5feb2fba72 roles = RoleSet { roles: {} } uri = /metrics/producers
39542024-08-28T02:19:03.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64134 req_id = 46623a08-8755-4840-b54a-5f5feb2fba72 resource = Database result = Ok(()) uri = /metrics/producers
39552024-08-28T02:19:03.738ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
39562024-08-28T02:19:03.738ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97e487f4-776f-4925-a724-69bd2ac937ca", "date": "Wed, 28 Aug 2024 02:19:03 GMT"} })
39572024-08-28T02:19:03.738ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = f5d97867-5abf-481d-a256-03ebe539fcd7
39582024-08-28T02:19:03.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 73181 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64134 req_id = 46623a08-8755-4840-b54a-5f5feb2fba72 response_code = 201 uri = /metrics/producers
39592024-08-28T02:19:04.201ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7e633248-55a0-4813-bfb1-2475359f0e03 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39602024-08-28T02:19:04.201ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7e633248-55a0-4813-bfb1-2475359f0e03 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39612024-08-28T02:19:04.203ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2895 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 7e633248-55a0-4813-bfb1-2475359f0e03 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39622024-08-28T02:19:05.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b3620cf2-55a0-459f-aed1-1ec8c294c24c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39632024-08-28T02:19:05.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b3620cf2-55a0-459f-aed1-1ec8c294c24c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39642024-08-28T02:19:05.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2214 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b3620cf2-55a0-459f-aed1-1ec8c294c24c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39652024-08-28T02:19:05.778ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 3074e2cf-d702-4411-baee-f7bf9f833ab2 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
39662024-08-28T02:19:05.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 3074e2cf-d702-4411-baee-f7bf9f833ab2 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
39672024-08-28T02:19:05.781ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3490 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 3074e2cf-d702-4411-baee-f7bf9f833ab2 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
39682024-08-28T02:19:05.989ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1643 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 9f77b2f4-0975-4256-8b19-24a140a757de response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
39692024-08-28T02:19:05.989ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
39702024-08-28T02:19:05.989ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
39712024-08-28T02:19:06.020ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
39722024-08-28T02:19:06.020ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:51707
39732024-08-28T02:19:06.020ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51707 req_id = db1e3098-8820-4808-9d41-c49db4b7b25e roles = RoleSet { roles: {} } uri = /metrics/producers
39742024-08-28T02:19:06.021ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51707 req_id = db1e3098-8820-4808-9d41-c49db4b7b25e resource = Database result = Ok(()) uri = /metrics/producers
39752024-08-28T02:19:06.054ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
39762024-08-28T02:19:06.054ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51707 req_id = db1e3098-8820-4808-9d41-c49db4b7b25e roles = RoleSet { roles: {} } uri = /metrics/producers
39772024-08-28T02:19:06.054ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51707 req_id = db1e3098-8820-4808-9d41-c49db4b7b25e resource = Database result = Ok(()) uri = /metrics/producers
39782024-08-28T02:19:06.061ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
39792024-08-28T02:19:06.061ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "16a333ab-eede-4cdb-a3aa-6de257d2a986", "date": "Wed, 28 Aug 2024 02:19:06 GMT"} })
39802024-08-28T02:19:06.061ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 50ac3e36-ea5c-4742-b23f-aa0a78804739
39812024-08-28T02:19:06.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 41943 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51707 req_id = db1e3098-8820-4808-9d41-c49db4b7b25e response_code = 201 uri = /metrics/producers
39822024-08-28T02:19:06.062ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "db1e3098-8820-4808-9d41-c49db4b7b25e", "content-length": "41", "date": "Wed, 28 Aug 2024 02:19:05 GMT"} })
39832024-08-28T02:19:06.063ZDEBGnexus (producer-registration-task): registered with nexus successfully
39842024-08-28T02:19:06.063ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
39852024-08-28T02:19:06.243ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c3a6df39-17a5-418b-92a8-ba2cbb655a9e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39862024-08-28T02:19:06.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c3a6df39-17a5-418b-92a8-ba2cbb655a9e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39872024-08-28T02:19:06.245ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = c3a6df39-17a5-418b-92a8-ba2cbb655a9e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
39882024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
39892024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
39902024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39912024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39922024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39932024-08-28T02:19:07.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39942024-08-28T02:19:07.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39952024-08-28T02:19:07.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39962024-08-28T02:19:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39972024-08-28T02:19:07.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.495609ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39982024-08-28T02:19:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39992024-08-28T02:19:07.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40002024-08-28T02:19:07.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40012024-08-28T02:19:07.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40022024-08-28T02:19:07.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40032024-08-28T02:19:07.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40042024-08-28T02:19:07.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
40052024-08-28T02:19:07.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40062024-08-28T02:19:07.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40072024-08-28T02:19:07.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
40082024-08-28T02:19:07.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.879615ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
40092024-08-28T02:19:07.265ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b399970f-95d4-4174-842a-08e4d6a0ed4c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40102024-08-28T02:19:07.265ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b399970f-95d4-4174-842a-08e4d6a0ed4c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40112024-08-28T02:19:07.267ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b399970f-95d4-4174-842a-08e4d6a0ed4c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40122024-08-28T02:19:08.286ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8882ddfc-9897-4afd-908d-a6aeb2a45878 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40132024-08-28T02:19:08.287ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8882ddfc-9897-4afd-908d-a6aeb2a45878 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40142024-08-28T02:19:08.288ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8882ddfc-9897-4afd-908d-a6aeb2a45878 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40152024-08-28T02:19:09.307ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4417d82c-ea2c-4927-bdf9-671ad234a543 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40162024-08-28T02:19:09.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4417d82c-ea2c-4927-bdf9-671ad234a543 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40172024-08-28T02:19:09.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1781 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4417d82c-ea2c-4927-bdf9-671ad234a543 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40182024-08-28T02:19:10.326ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 65daa612-0536-4c47-bde4-615770cb7d49 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40192024-08-28T02:19:10.327ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 65daa612-0536-4c47-bde4-615770cb7d49 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40202024-08-28T02:19:10.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2128 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 65daa612-0536-4c47-bde4-615770cb7d49 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40212024-08-28T02:19:11.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2ab38127-2499-47ab-b590-16d51ccb0090 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40222024-08-28T02:19:11.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2ab38127-2499-47ab-b590-16d51ccb0090 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40232024-08-28T02:19:11.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 2ab38127-2499-47ab-b590-16d51ccb0090 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40242024-08-28T02:19:11.404ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64169
40252024-08-28T02:19:11.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64169 req_id = b37a8f34-a8c4-4bfc-8a14-cf592b82c94e roles = RoleSet { roles: {} } uri = /metrics/producers
40262024-08-28T02:19:11.405ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64169 req_id = b37a8f34-a8c4-4bfc-8a14-cf592b82c94e resource = Database result = Ok(()) uri = /metrics/producers
40272024-08-28T02:19:11.438ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
40282024-08-28T02:19:11.438ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64169 req_id = b37a8f34-a8c4-4bfc-8a14-cf592b82c94e roles = RoleSet { roles: {} } uri = /metrics/producers
40292024-08-28T02:19:11.438ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64169 req_id = b37a8f34-a8c4-4bfc-8a14-cf592b82c94e resource = Database result = Ok(()) uri = /metrics/producers
40302024-08-28T02:19:11.474ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
40312024-08-28T02:19:11.475ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d91e46f0-405c-4cc7-b19f-9cf6f69b6f2e", "date": "Wed, 28 Aug 2024 02:19:11 GMT"} })
40322024-08-28T02:19:11.475ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 7ab6cf11-bb3f-4b6e-b7eb-0005add95826
40332024-08-28T02:19:11.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 71677 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64169 req_id = b37a8f34-a8c4-4bfc-8a14-cf592b82c94e response_code = 201 uri = /metrics/producers
40342024-08-28T02:19:11.815ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:37412
40352024-08-28T02:19:11.815ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37412 req_id = a20d4fb7-4c02-4113-a87f-062ddb57ea97 roles = RoleSet { roles: {} } uri = /metrics/producers
40362024-08-28T02:19:11.815ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37412 req_id = a20d4fb7-4c02-4113-a87f-062ddb57ea97 resource = Database result = Ok(()) uri = /metrics/producers
40372024-08-28T02:19:11.849ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
40382024-08-28T02:19:11.849ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37412 req_id = a20d4fb7-4c02-4113-a87f-062ddb57ea97 roles = RoleSet { roles: {} } uri = /metrics/producers
40392024-08-28T02:19:11.849ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37412 req_id = a20d4fb7-4c02-4113-a87f-062ddb57ea97 resource = Database result = Ok(()) uri = /metrics/producers
40402024-08-28T02:19:11.879ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
40412024-08-28T02:19:11.879ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "618af56e-6bf3-4338-a551-afd9b7916e22", "date": "Wed, 28 Aug 2024 02:19:11 GMT"} })
40422024-08-28T02:19:11.879ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 86d235a7-2247-4a0d-a3ed-09ecc4c92890
40432024-08-28T02:19:11.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 66332 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:37412 req_id = a20d4fb7-4c02-4113-a87f-062ddb57ea97 response_code = 201 uri = /metrics/producers
40442024-08-28T02:19:12.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54f04a81-f80a-4544-a477-8badb980d1a4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40452024-08-28T02:19:12.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54f04a81-f80a-4544-a477-8badb980d1a4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40462024-08-28T02:19:12.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1989 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 54f04a81-f80a-4544-a477-8badb980d1a4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40472024-08-28T02:19:13.388ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9cf3b1a-00de-4437-bc44-3892fc704287 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40482024-08-28T02:19:13.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9cf3b1a-00de-4437-bc44-3892fc704287 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40492024-08-28T02:19:13.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2598 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b9cf3b1a-00de-4437-bc44-3892fc704287 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40502024-08-28T02:19:14.410ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b744fd9-b9dc-41d4-a996-427058044c1c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40512024-08-28T02:19:14.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b744fd9-b9dc-41d4-a996-427058044c1c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40522024-08-28T02:19:14.412ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2313 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6b744fd9-b9dc-41d4-a996-427058044c1c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40532024-08-28T02:19:15.431ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5d39f24-fd9f-48f8-a8cf-f298fd3c3e73 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40542024-08-28T02:19:15.432ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5d39f24-fd9f-48f8-a8cf-f298fd3c3e73 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40552024-08-28T02:19:15.433ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1978 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = b5d39f24-fd9f-48f8-a8cf-f298fd3c3e73 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40562024-08-28T02:19:16.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1589 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = d6e3edd3-eb14-4d5b-86c7-8b9f7bb4ab85 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
40572024-08-28T02:19:16.453ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a72ecd31-bbee-4af8-9226-a9bdc15e11ae roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40582024-08-28T02:19:16.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a72ecd31-bbee-4af8-9226-a9bdc15e11ae resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40592024-08-28T02:19:16.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2116 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a72ecd31-bbee-4af8-9226-a9bdc15e11ae response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
40602024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
40612024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
40622024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
40632024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
40642024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40652024-08-28T02:19:17.145ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758
40662024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.68\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758"}
40672024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
40682024-08-28T02:19:17.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
40692024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
40702024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40712024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
40722024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
40732024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40742024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40752024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
40762024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
40772024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40782024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
40792024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40802024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
40812024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
40822024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40832024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
40842024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40852024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
40862024-08-28T02:19:17.145ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
40872024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
40882024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
40892024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40902024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
40912024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
40922024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
40932024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
40942024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40952024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
40962024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40972024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
40982024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
40992024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41002024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
41012024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
41022024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
41032024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41042024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
41052024-08-28T02:19:17.145ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-28 02:09:17.145704891 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
41062024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
41072024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
41082024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41092024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
41102024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
41112024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41122024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
41132024-08-28T02:19:17.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
41142024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
41152024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
41162024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
41172024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
41182024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41192024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41202024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41212024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41222024-08-28T02:19:17.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41232024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41242024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41252024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41262024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
41272024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41282024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41292024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41302024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41312024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41322024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41332024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41342024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41352024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41362024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
41372024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41382024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41392024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41402024-08-28T02:19:17.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
41412024-08-28T02:19:17.147ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("01868911-e8e4-4a77-b105-b83672ce13b2.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c854ff-da18-4c2a-9d2f-7e15248c76a8.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f85a8e69-10b1-4e9c-982f-31422be2a907.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1365, tv_nsec: 305231004 } })
41422024-08-28T02:19:17.147ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6926ac43-4e2e-4378-9bce-42019ec72a30.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("7a899aef-896e-41b9-8810-6c5024da5a0c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1365, tv_nsec: 305243035 } })
41432024-08-28T02:19:17.149ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
41442024-08-28T02:19:17.149ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.565531ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
41452024-08-28T02:19:17.149ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41462024-08-28T02:19:17.149ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41472024-08-28T02:19:17.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41482024-08-28T02:19:17.150ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.51822ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41492024-08-28T02:19:17.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41502024-08-28T02:19:17.150ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
41512024-08-28T02:19:17.150ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.721964ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
41522024-08-28T02:19:17.150ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1365, tv_nsec: 308329466 } })
41532024-08-28T02:19:17.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
41542024-08-28T02:19:17.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1365, tv_nsec: 309470639 } })
41552024-08-28T02:19:17.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41562024-08-28T02:19:17.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41572024-08-28T02:19:17.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41582024-08-28T02:19:17.152ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
41592024-08-28T02:19:17.152ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41602024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41612024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
41622024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41632024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41642024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41652024-08-28T02:19:17.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41662024-08-28T02:19:17.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41672024-08-28T02:19:17.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41682024-08-28T02:19:17.154ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 8.673372ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
41692024-08-28T02:19:17.154ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41702024-08-28T02:19:17.155ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41712024-08-28T02:19:17.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
41722024-08-28T02:19:17.156ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41732024-08-28T02:19:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41742024-08-28T02:19:17.157ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41752024-08-28T02:19:17.157ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41762024-08-28T02:19:17.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41772024-08-28T02:19:17.158ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41782024-08-28T02:19:17.158ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41792024-08-28T02:19:17.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41802024-08-28T02:19:17.160ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-28 02:09:17.145704891 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
41812024-08-28T02:19:17.160ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.357095ms iteration = 9 status = {"expiration":"2024-08-28T02:09:17.145704891Z","pruned":[]}
41822024-08-28T02:19:17.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41832024-08-28T02:19:17.160ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41842024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41852024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41862024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41872024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41882024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41892024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41902024-08-28T02:19:17.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41912024-08-28T02:19:17.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41922024-08-28T02:19:17.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41932024-08-28T02:19:17.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41942024-08-28T02:19:17.166ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41952024-08-28T02:19:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41962024-08-28T02:19:17.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41972024-08-28T02:19:17.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41982024-08-28T02:19:17.167ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41992024-08-28T02:19:17.168ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 22.972456ms iteration = 17 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
42002024-08-28T02:19:17.168ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42012024-08-28T02:19:17.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42022024-08-28T02:19:17.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42032024-08-28T02:19:17.169ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42042024-08-28T02:19:17.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42052024-08-28T02:19:17.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42062024-08-28T02:19:17.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42072024-08-28T02:19:17.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42082024-08-28T02:19:17.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42092024-08-28T02:19:17.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42102024-08-28T02:19:17.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42112024-08-28T02:19:17.174ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42122024-08-28T02:19:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42132024-08-28T02:19:17.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42142024-08-28T02:19:17.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42152024-08-28T02:19:17.177ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1365, tv_nsec: 335872313 } })
42162024-08-28T02:19:17.178ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
42172024-08-28T02:19:17.178ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 32.507785ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
42182024-08-28T02:19:17.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
42192024-08-28T02:19:17.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42202024-08-28T02:19:17.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42212024-08-28T02:19:17.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42222024-08-28T02:19:17.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42232024-08-28T02:19:17.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
42242024-08-28T02:19:17.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42252024-08-28T02:19:17.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42262024-08-28T02:19:17.190ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
42272024-08-28T02:19:17.190ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42282024-08-28T02:19:17.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42292024-08-28T02:19:17.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42302024-08-28T02:19:17.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42312024-08-28T02:19:17.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42322024-08-28T02:19:17.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42332024-08-28T02:19:17.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42342024-08-28T02:19:17.196ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
42352024-08-28T02:19:17.196ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
42362024-08-28T02:19:17.196ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 50.35498ms iteration = 9 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
42372024-08-28T02:19:17.201ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.775697ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
42382024-08-28T02:19:17.205ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
42392024-08-28T02:19:17.205ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 60.44948ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
42402024-08-28T02:19:17.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.132634ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42412024-08-28T02:19:17.209ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
42422024-08-28T02:19:17.209ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
42432024-08-28T02:19:17.209ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 64.009671ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
42442024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
42452024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
42462024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
42472024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
42482024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
42492024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42502024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
42512024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42522024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42532024-08-28T02:19:17.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42542024-08-28T02:19:17.220ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42552024-08-28T02:19:17.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42562024-08-28T02:19:17.282ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
42572024-08-28T02:19:17.321ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
42582024-08-28T02:19:17.339ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
42592024-08-28T02:19:17.339ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42602024-08-28T02:19:17.340ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "54c43c1c-5de4-4eb0-bccb-9b3ff5b83a7a", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
42612024-08-28T02:19:17.340ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42622024-08-28T02:19:17.341ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
42632024-08-28T02:19:17.389ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42642024-08-28T02:19:17.390ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "01fe011c-2076-4a75-9fea-e542c0f155d6", "content-length": "372", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} })
42652024-08-28T02:19:17.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42662024-08-28T02:19:17.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42672024-08-28T02:19:17.391ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42682024-08-28T02:19:17.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42692024-08-28T02:19:17.393ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42702024-08-28T02:19:17.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42712024-08-28T02:19:17.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42722024-08-28T02:19:17.394ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42732024-08-28T02:19:17.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42742024-08-28T02:19:17.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
42752024-08-28T02:19:17.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42762024-08-28T02:19:17.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42772024-08-28T02:19:17.397ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42782024-08-28T02:19:17.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42792024-08-28T02:19:17.397ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42802024-08-28T02:19:17.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42812024-08-28T02:19:17.399ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42822024-08-28T02:19:17.399ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42832024-08-28T02:19:17.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42842024-08-28T02:19:17.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42852024-08-28T02:19:17.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42862024-08-28T02:19:17.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
42872024-08-28T02:19:17.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42882024-08-28T02:19:17.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42892024-08-28T02:19:17.404ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 258.691407ms iteration = 17 status = 0
42902024-08-28T02:19:17.404ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42912024-08-28T02:19:17.405ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "34900b97-dac1-48db-aa42-d78122063122", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} })
42922024-08-28T02:19:17.406ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
42932024-08-28T02:19:17.406ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 260.736918ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
42942024-08-28T02:19:17.406ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 261.402071ms iteration = 19 status = {}
42952024-08-28T02:19:17.419ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42962024-08-28T02:19:17.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
42972024-08-28T02:19:17.419ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 274.160844ms iteration = 17 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
42982024-08-28T02:19:17.421ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42992024-08-28T02:19:17.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43002024-08-28T02:19:17.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43012024-08-28T02:19:17.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43022024-08-28T02:19:17.425ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43032024-08-28T02:19:17.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43042024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
43052024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43062024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43072024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
43082024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43092024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
43102024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 0 version = 2
43112024-08-28T02:19:17.428ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43122024-08-28T02:19:17.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
43132024-08-28T02:19:17.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43142024-08-28T02:19:17.429ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
43152024-08-28T02:19:17.429ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
43162024-08-28T02:19:17.429ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 283.700274ms iteration = 17 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
43172024-08-28T02:19:17.435ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43182024-08-28T02:19:17.438ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43192024-08-28T02:19:17.438ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43202024-08-28T02:19:17.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43212024-08-28T02:19:17.441ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
43222024-08-28T02:19:17.443ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0331076d-c0d4-495d-92c4-0eed38567238", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43232024-08-28T02:19:17.443ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43242024-08-28T02:19:17.445ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9653ea3c-95ce-4608-ba4d-2ed46147eafd", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43252024-08-28T02:19:17.445ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43262024-08-28T02:19:17.446ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
43272024-08-28T02:19:17.446ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 301.358304ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
43282024-08-28T02:19:17.448ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
43292024-08-28T02:19:17.448ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43302024-08-28T02:19:17.449ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b8efdaf9-48df-4477-a90a-2188713611ec", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
43312024-08-28T02:19:17.449ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43322024-08-28T02:19:17.451ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43332024-08-28T02:19:17.456ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43342024-08-28T02:19:17.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43352024-08-28T02:19:17.459ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43362024-08-28T02:19:17.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43372024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60636cd6-39ac-4391-ab23-4b1259310aca dns_zone_name = oxide.test found_so_far = 1 version = 2
43382024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
43392024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-28 02:11:35.148647 UTC dns_group = external generation = 2
43402024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 318.099187ms iteration = 10 status = {"generation":2}
43412024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
43422024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
43432024-08-28T02:19:17.463ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43442024-08-28T02:19:17.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43452024-08-28T02:19:17.464ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
43462024-08-28T02:19:17.464ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43472024-08-28T02:19:17.464ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "da1fb683-89c2-4331-9975-23236564d43f", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
43482024-08-28T02:19:17.464ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43492024-08-28T02:19:17.466ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43502024-08-28T02:19:17.467ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-28T02:11:15.536604Z, time_modified: 2024-08-28T02:11:15.536604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, db_silo: Silo { identity: SiloIdentity { id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-28T02:11:35.100950Z, time_modified: 2024-08-28T02:11:35.100950Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac" }] }) }
43512024-08-28T02:19:17.467ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 322.016585ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]}},"default_endpoint":{"silo_id":"bb0007f2-a695-4aac-b57b-8cca99a33c5b","tls_certs":["71daeb7010610692abc666e6934ec592c263cfc15250f503454ed4234cc46cac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
43522024-08-28T02:19:17.478ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
43532024-08-28T02:19:17.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8bbd81de-d0fc-451c-a7b6-b2935efc34cc roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
43542024-08-28T02:19:17.479ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8bbd81de-d0fc-451c-a7b6-b2935efc34cc resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
43552024-08-28T02:19:17.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8bbd81de-d0fc-451c-a7b6-b2935efc34cc response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
43562024-08-28T02:19:17.489ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 343.404829ms iteration = 20 status = {"physical_disks_added":0}
43572024-08-28T02:19:17.492ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43582024-08-28T02:19:17.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
43592024-08-28T02:19:17.495ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43602024-08-28T02:19:17.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43612024-08-28T02:19:17.498ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = aba0906a-ff7e-41a8-9f51-7349b309c33b dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
43622024-08-28T02:19:17.498ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
43632024-08-28T02:19:17.498ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-28 02:11:34.295733 UTC dns_group = internal generation = 1
43642024-08-28T02:19:17.498ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 352.455779ms iteration = 10 status = {"generation":1}
43652024-08-28T02:19:17.585ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
43662024-08-28T02:19:17.585ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "42b16888-1514-4b83-afe8-4f6473740cf9", "content-length": "2", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
43672024-08-28T02:19:17.586ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43682024-08-28T02:19:17.587ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43692024-08-28T02:19:17.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43702024-08-28T02:19:17.587ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a19f5b5e-e1d8-489f-b647-6e872f270b49", "content-length": "691", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} })
43712024-08-28T02:19:17.587ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
43722024-08-28T02:19:17.587ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
43732024-08-28T02:19:17.587ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
43742024-08-28T02:19:17.589ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 443.910475ms iteration = 17 status = null
43752024-08-28T02:19:17.589ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 444.229931ms iteration = 18 status = {}
43762024-08-28T02:19:17.592ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 447.470536ms iteration = 17 status = {}
43772024-08-28T02:19:17.599ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
43782024-08-28T02:19:17.600ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0f19b0a4-f17f-456b-8c12-f47e2644666a", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43792024-08-28T02:19:17.600ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43802024-08-28T02:19:17.601ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "091911b8-1216-4ede-b649-beea641f54db", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43812024-08-28T02:19:17.601ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43822024-08-28T02:19:17.602ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6bca53fc-a675-4b68-a42b-5d8772691e93", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43832024-08-28T02:19:17.602ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43842024-08-28T02:19:17.604ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
43852024-08-28T02:19:17.604ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 458.901632ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
43862024-08-28T02:19:17.630ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43872024-08-28T02:19:17.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43882024-08-28T02:19:17.632ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
43892024-08-28T02:19:17.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43902024-08-28T02:19:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43912024-08-28T02:19:17.642ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
43922024-08-28T02:19:17.642ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1f2cdeac-acf2-478d-9de0-5c27fc64ceae", "content-length": "121", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
43932024-08-28T02:19:17.642ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
43942024-08-28T02:19:17.642ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
43952024-08-28T02:19:17.643ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43962024-08-28T02:19:17.643ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43972024-08-28T02:19:17.643ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
43982024-08-28T02:19:17.643ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
43992024-08-28T02:19:17.643ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
44002024-08-28T02:19:17.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44012024-08-28T02:19:17.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44022024-08-28T02:19:17.645ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
44032024-08-28T02:19:17.645ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b46267d-f801-454b-b54d-bff60eaea034", "content-length": "2", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
44042024-08-28T02:19:17.645ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
44052024-08-28T02:19:17.646ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "21c4a91b-1e2a-46c1-a697-2dd2f36b9727", "content-length": "51", "date": "Wed, 28 Aug 2024 02:19:17 GMT"} })
44062024-08-28T02:19:17.646ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
44072024-08-28T02:19:17.646ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
44082024-08-28T02:19:17.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c91191fa-3f89-460d-9434-ed074f3cb90a", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} })
44092024-08-28T02:19:17.730ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
44102024-08-28T02:19:17.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
44112024-08-28T02:19:17.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5187d404-b25d-4f31-9302-3cf27b393cb2", "content-length": "667", "date": "Wed, 28 Aug 2024 02:19:16 GMT"} })
44122024-08-28T02:19:17.731ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44132024-08-28T02:19:17.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44142024-08-28T02:19:17.734ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44152024-08-28T02:19:17.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44162024-08-28T02:19:17.738ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44172024-08-28T02:19:17.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44182024-08-28T02:19:17.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44192024-08-28T02:19:17.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44202024-08-28T02:19:17.747ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 602.170737ms iteration = 17 status = {}
44212024-08-28T02:19:18.501ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc5bdd69-3fd9-4a9d-9857-cefbe5197d3c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44222024-08-28T02:19:18.502ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc5bdd69-3fd9-4a9d-9857-cefbe5197d3c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44232024-08-28T02:19:18.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2674 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = dc5bdd69-3fd9-4a9d-9857-cefbe5197d3c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44242024-08-28T02:19:19.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f867f209-9d47-4dbe-bf24-6439abd82eef roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44252024-08-28T02:19:19.523ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f867f209-9d47-4dbe-bf24-6439abd82eef resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44262024-08-28T02:19:19.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2193 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f867f209-9d47-4dbe-bf24-6439abd82eef response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44272024-08-28T02:19:20.236ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:50189
44282024-08-28T02:19:20.236ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:50189 req_id = fe8bb5d3-e24c-462c-815b-17488529fbda roles = RoleSet { roles: {} } uri = /metrics/producers
44292024-08-28T02:19:20.236ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:50189 req_id = fe8bb5d3-e24c-462c-815b-17488529fbda resource = Database result = Ok(()) uri = /metrics/producers
44302024-08-28T02:19:20.270ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841
44312024-08-28T02:19:20.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:50189 req_id = fe8bb5d3-e24c-462c-815b-17488529fbda roles = RoleSet { roles: {} } uri = /metrics/producers
44322024-08-28T02:19:20.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:50189 req_id = fe8bb5d3-e24c-462c-815b-17488529fbda resource = Database result = Ok(()) uri = /metrics/producers
44332024-08-28T02:19:20.302ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 uri = http://[fd00:1122:3344:101::d]:12223/producers
44342024-08-28T02:19:20.302ZDEBGnexus: client response oximeter-collector = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fd46c7cd-c6fe-4a3c-b631-6787439d692b", "date": "Wed, 28 Aug 2024 02:19:20 GMT"} })
44352024-08-28T02:19:20.302ZINFOnexus: assigned collector to new producer collector_id = 1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = nexus/src/app/oximeter.rs:125 producer_id = 92e923a4-5690-44dd-9005-892bead54a2c
44362024-08-28T02:19:20.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 67367 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:50189 req_id = fe8bb5d3-e24c-462c-815b-17488529fbda response_code = 201 uri = /metrics/producers
44372024-08-28T02:19:20.544ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fd136cb1-4752-43d4-bd7d-e9a5777c130b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44382024-08-28T02:19:20.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fd136cb1-4752-43d4-bd7d-e9a5777c130b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44392024-08-28T02:19:20.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2670 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fd136cb1-4752-43d4-bd7d-e9a5777c130b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44402024-08-28T02:19:21.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e996bc8a-8d5b-4434-a579-eec0c9c7e5ca roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44412024-08-28T02:19:21.567ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e996bc8a-8d5b-4434-a579-eec0c9c7e5ca resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44422024-08-28T02:19:21.569ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2882 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = e996bc8a-8d5b-4434-a579-eec0c9c7e5ca response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44432024-08-28T02:19:22.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e9eb610-73f5-43f3-8a09-d60653e46107 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44442024-08-28T02:19:22.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e9eb610-73f5-43f3-8a09-d60653e46107 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44452024-08-28T02:19:22.588ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1888 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 1e9eb610-73f5-43f3-8a09-d60653e46107 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44462024-08-28T02:19:23.606ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8852e048-322c-4823-81a1-968c675ccec4 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44472024-08-28T02:19:23.607ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8852e048-322c-4823-81a1-968c675ccec4 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44482024-08-28T02:19:23.609ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2593 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8852e048-322c-4823-81a1-968c675ccec4 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44492024-08-28T02:19:24.632ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 027667e4-a9ac-4595-92ae-4d5efe113073 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44502024-08-28T02:19:24.632ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 027667e4-a9ac-4595-92ae-4d5efe113073 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44512024-08-28T02:19:24.634ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2862 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 027667e4-a9ac-4595-92ae-4d5efe113073 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44522024-08-28T02:19:25.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff9735f1-07fe-4688-934a-0cc868d6da72 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44532024-08-28T02:19:25.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff9735f1-07fe-4688-934a-0cc868d6da72 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44542024-08-28T02:19:25.657ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2715 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff9735f1-07fe-4688-934a-0cc868d6da72 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44552024-08-28T02:19:26.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1906 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 9edd3f05-a23a-4898-b451-b96a95396a5f response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
44562024-08-28T02:19:26.677ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f324caf9-a74f-48ff-862d-5ce3cfdad923 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44572024-08-28T02:19:26.677ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f324caf9-a74f-48ff-862d-5ce3cfdad923 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44582024-08-28T02:19:26.679ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2537 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f324caf9-a74f-48ff-862d-5ce3cfdad923 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44592024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
44602024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
44612024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44622024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44632024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44642024-08-28T02:19:27.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44652024-08-28T02:19:27.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44662024-08-28T02:19:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44672024-08-28T02:19:27.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44682024-08-28T02:19:27.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44692024-08-28T02:19:27.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44702024-08-28T02:19:27.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44712024-08-28T02:19:27.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44722024-08-28T02:19:27.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
44732024-08-28T02:19:27.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44742024-08-28T02:19:27.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44752024-08-28T02:19:27.166ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
44762024-08-28T02:19:27.166ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.351784ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
44772024-08-28T02:19:27.173ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44782024-08-28T02:19:27.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44792024-08-28T02:19:27.175ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.158629ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44802024-08-28T02:19:27.698ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 366e4093-b321-4845-b6c2-4940c7fc471d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44812024-08-28T02:19:27.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 366e4093-b321-4845-b6c2-4940c7fc471d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44822024-08-28T02:19:27.700ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2554 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 366e4093-b321-4845-b6c2-4940c7fc471d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44832024-08-28T02:19:28.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac8977e9-0945-4253-8d04-adbf55cbe320 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44842024-08-28T02:19:28.720ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac8977e9-0945-4253-8d04-adbf55cbe320 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44852024-08-28T02:19:28.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2815 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ac8977e9-0945-4253-8d04-adbf55cbe320 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44862024-08-28T02:19:29.739ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 59b0b4e2-48a7-4fd7-934c-d7c37cb247d9 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44872024-08-28T02:19:29.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 59b0b4e2-48a7-4fd7-934c-d7c37cb247d9 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44882024-08-28T02:19:29.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2194 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 59b0b4e2-48a7-4fd7-934c-d7c37cb247d9 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44892024-08-28T02:19:30.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 511358d4-aca0-4008-9861-962b68bb1537 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44902024-08-28T02:19:30.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 511358d4-aca0-4008-9861-962b68bb1537 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44912024-08-28T02:19:30.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2046 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 511358d4-aca0-4008-9861-962b68bb1537 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44922024-08-28T02:19:31.781ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f3af6884-1906-4071-8c4e-c08f49ce443c roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44932024-08-28T02:19:31.781ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f3af6884-1906-4071-8c4e-c08f49ce443c resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44942024-08-28T02:19:31.782ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1950 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = f3af6884-1906-4071-8c4e-c08f49ce443c response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44952024-08-28T02:19:32.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ddf6bf4-28cf-4c61-b239-44920a2c8606 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44962024-08-28T02:19:32.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ddf6bf4-28cf-4c61-b239-44920a2c8606 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44972024-08-28T02:19:32.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2297 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3ddf6bf4-28cf-4c61-b239-44920a2c8606 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44982024-08-28T02:19:33.823ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce570fda-63a0-40f5-84ec-d6709243276f roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
44992024-08-28T02:19:33.823ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce570fda-63a0-40f5-84ec-d6709243276f resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45002024-08-28T02:19:33.825ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2297 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ce570fda-63a0-40f5-84ec-d6709243276f response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45012024-08-28T02:19:34.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03a3faff-88c7-40bf-a41f-54923cbffa93 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45022024-08-28T02:19:34.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03a3faff-88c7-40bf-a41f-54923cbffa93 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45032024-08-28T02:19:34.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 03a3faff-88c7-40bf-a41f-54923cbffa93 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45042024-08-28T02:19:35.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 02aa5cc2-fe54-4a93-83f1-6ae4bd8b8ff6 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
45052024-08-28T02:19:35.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 02aa5cc2-fe54-4a93-83f1-6ae4bd8b8ff6 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
45062024-08-28T02:19:35.815ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3219 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:40240 req_id = 02aa5cc2-fe54-4a93-83f1-6ae4bd8b8ff6 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
45072024-08-28T02:19:35.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = afa97dc9-721f-49eb-ba8c-0c67968cf6c7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45082024-08-28T02:19:35.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = afa97dc9-721f-49eb-ba8c-0c67968cf6c7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45092024-08-28T02:19:35.866ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1833 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = afa97dc9-721f-49eb-ba8c-0c67968cf6c7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45102024-08-28T02:19:36.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1519 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 625bf33f-5a88-4c88-8fe3-5c40c19bf8a7 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
45112024-08-28T02:19:36.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1211938-7498-4d4e-86b2-d98d3be9afb2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45122024-08-28T02:19:36.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1211938-7498-4d4e-86b2-d98d3be9afb2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45132024-08-28T02:19:36.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2278 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = d1211938-7498-4d4e-86b2-d98d3be9afb2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45142024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
45152024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
45162024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45172024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45182024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45192024-08-28T02:19:37.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45202024-08-28T02:19:37.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45212024-08-28T02:19:37.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45222024-08-28T02:19:37.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45232024-08-28T02:19:37.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.270795ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45242024-08-28T02:19:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45252024-08-28T02:19:37.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45262024-08-28T02:19:37.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45272024-08-28T02:19:37.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45282024-08-28T02:19:37.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45292024-08-28T02:19:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45302024-08-28T02:19:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
45312024-08-28T02:19:37.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45322024-08-28T02:19:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45332024-08-28T02:19:37.163ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
45342024-08-28T02:19:37.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.582099ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
45352024-08-28T02:19:37.905ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff4a090f-dcc5-4a17-a070-c229c5cdbbb6 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45362024-08-28T02:19:37.906ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff4a090f-dcc5-4a17-a070-c229c5cdbbb6 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45372024-08-28T02:19:37.907ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2185 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = ff4a090f-dcc5-4a17-a070-c229c5cdbbb6 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45382024-08-28T02:19:38.929ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71da1d83-1bec-470d-a09d-0735da7d3e6e roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45392024-08-28T02:19:38.929ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71da1d83-1bec-470d-a09d-0735da7d3e6e resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45402024-08-28T02:19:38.931ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2256 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 71da1d83-1bec-470d-a09d-0735da7d3e6e response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45412024-08-28T02:19:39.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1077837-7c97-417c-b80e-8883600f93b3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45422024-08-28T02:19:39.950ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1077837-7c97-417c-b80e-8883600f93b3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45432024-08-28T02:19:39.952ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = a1077837-7c97-417c-b80e-8883600f93b3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45442024-08-28T02:19:40.972ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4818dba4-7452-463f-847b-c430c218c359 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45452024-08-28T02:19:40.972ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4818dba4-7452-463f-847b-c430c218c359 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45462024-08-28T02:19:40.974ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2502 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4818dba4-7452-463f-847b-c430c218c359 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45472024-08-28T02:19:41.995ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5ab05f-b95b-46a0-b7c3-afaf198a8228 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45482024-08-28T02:19:41.995ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5ab05f-b95b-46a0-b7c3-afaf198a8228 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45492024-08-28T02:19:41.997ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2380 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3a5ab05f-b95b-46a0-b7c3-afaf198a8228 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45502024-08-28T02:19:43.017ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 57c26f12-0613-4805-ae95-df197aa04af8 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45512024-08-28T02:19:43.018ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 57c26f12-0613-4805-ae95-df197aa04af8 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45522024-08-28T02:19:43.020ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2952 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 57c26f12-0613-4805-ae95-df197aa04af8 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45532024-08-28T02:19:44.038ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ba7b1be-783d-49b8-bbc6-93f7b1b032d0 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45542024-08-28T02:19:44.038ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ba7b1be-783d-49b8-bbc6-93f7b1b032d0 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45552024-08-28T02:19:44.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2653 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 4ba7b1be-783d-49b8-bbc6-93f7b1b032d0 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45562024-08-28T02:19:45.059ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84d277bb-1acc-463c-af46-79fca16b05e3 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45572024-08-28T02:19:45.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84d277bb-1acc-463c-af46-79fca16b05e3 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45582024-08-28T02:19:45.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1970 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 84d277bb-1acc-463c-af46-79fca16b05e3 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45592024-08-28T02:19:46.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2267 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 0eafaef0-992c-49aa-8c2a-5e6338af1adc response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
45602024-08-28T02:19:46.081ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0d885b82-f41c-4c43-a6a4-920f1d512768 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45612024-08-28T02:19:46.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0d885b82-f41c-4c43-a6a4-920f1d512768 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45622024-08-28T02:19:46.084ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2367 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 0d885b82-f41c-4c43-a6a4-920f1d512768 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45632024-08-28T02:19:47.103ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90f2aac6-00bd-415a-83a1-606189e0e070 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45642024-08-28T02:19:47.103ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90f2aac6-00bd-415a-83a1-606189e0e070 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45652024-08-28T02:19:47.105ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1955 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 90f2aac6-00bd-415a-83a1-606189e0e070 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
45662024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
45672024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
45682024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
45692024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45702024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45712024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
45722024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
45732024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
45742024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
45752024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
45762024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
45772024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45782024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
45792024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
45802024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45812024-08-28T02:19:47.145ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task started background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:417
45822024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45832024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
45842024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
45852024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45862024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
45872024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
45882024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45892024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
45902024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45912024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
45922024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45932024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
45942024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
45952024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
45962024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
45972024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45982024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
45992024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46002024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46012024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46022024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
46032024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46042024-08-28T02:19:47.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46052024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46062024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46072024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46082024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46092024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46102024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46112024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46122024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
46132024-08-28T02:19:47.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
46142024-08-28T02:19:47.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46152024-08-28T02:19:47.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46162024-08-28T02:19:47.149ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.833047ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
46172024-08-28T02:19:47.149ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1395, tv_nsec: 307700954 } })
46182024-08-28T02:19:47.149ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46192024-08-28T02:19:47.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
46202024-08-28T02:19:47.150ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46212024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
46222024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46232024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46242024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46252024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46262024-08-28T02:19:47.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46272024-08-28T02:19:47.152ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46282024-08-28T02:19:47.152ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46292024-08-28T02:19:47.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46302024-08-28T02:19:47.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
46312024-08-28T02:19:47.153ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46322024-08-28T02:19:47.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46332024-08-28T02:19:47.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46342024-08-28T02:19:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46352024-08-28T02:19:47.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46362024-08-28T02:19:47.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46372024-08-28T02:19:47.160ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1395, tv_nsec: 318082530 } })
46382024-08-28T02:19:47.162ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
46392024-08-28T02:19:47.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46402024-08-28T02:19:47.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46412024-08-28T02:19:47.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46422024-08-28T02:19:47.169ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-40b49400-5cd0-4d5c-a006-b241a1ca8261.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1395, tv_nsec: 327249052 } })
46432024-08-28T02:19:47.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46442024-08-28T02:19:47.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
46452024-08-28T02:19:47.189ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
46462024-08-28T02:19:47.189ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
46472024-08-28T02:19:47.189ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46482024-08-28T02:19:47.189ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 43.437803ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
46492024-08-28T02:19:47.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46502024-08-28T02:19:47.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46512024-08-28T02:19:47.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46522024-08-28T02:19:47.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46532024-08-28T02:19:47.196ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46542024-08-28T02:19:47.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46552024-08-28T02:19:47.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
46562024-08-28T02:19:47.197ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46572024-08-28T02:19:47.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46582024-08-28T02:19:47.197ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46592024-08-28T02:19:47.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46602024-08-28T02:19:47.200ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46612024-08-28T02:19:47.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46622024-08-28T02:19:47.201ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46632024-08-28T02:19:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46642024-08-28T02:19:47.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46652024-08-28T02:19:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46662024-08-28T02:19:47.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46672024-08-28T02:19:47.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46682024-08-28T02:19:47.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46692024-08-28T02:19:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46702024-08-28T02:19:47.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46712024-08-28T02:19:47.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46722024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
46732024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46742024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46752024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46762024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46772024-08-28T02:19:47.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46782024-08-28T02:19:47.218ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
46792024-08-28T02:19:47.218ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.314776ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
46802024-08-28T02:19:47.220ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 74.961129ms iteration = 21 status = {"physical_disks_added":0}
46812024-08-28T02:19:47.223ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46822024-08-28T02:19:47.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46832024-08-28T02:19:47.224ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
46842024-08-28T02:19:47.224ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46852024-08-28T02:19:47.224ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ef9ce9a8-8668-40ad-888e-88c74b876026", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
46862024-08-28T02:19:47.224ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46872024-08-28T02:19:47.226ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46882024-08-28T02:19:47.226ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46892024-08-28T02:19:47.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46902024-08-28T02:19:47.229ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46912024-08-28T02:19:47.232ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 86.921586ms iteration = 18 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
46922024-08-28T02:19:47.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 40b49400-5cd0-4d5c-a006-b241a1ca8261, lookup_type: ById(40b49400-5cd0-4d5c-a006-b241a1ca8261) } result = Ok(())
46932024-08-28T02:19:47.232ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46942024-08-28T02:19:47.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46952024-08-28T02:19:47.234ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
46962024-08-28T02:19:47.234ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
46972024-08-28T02:19:47.234ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 88.807223ms iteration = 18 status = 0
46982024-08-28T02:19:47.234ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 88.994027ms iteration = 18 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
46992024-08-28T02:19:47.235ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.660801ms iteration = 18 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
47002024-08-28T02:19:47.240ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47012024-08-28T02:19:47.240ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 94.64055ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47022024-08-28T02:19:47.241ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47032024-08-28T02:19:47.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47042024-08-28T02:19:47.241ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 96.076968ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47052024-08-28T02:19:47.243ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47062024-08-28T02:19:47.244ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47072024-08-28T02:19:47.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47082024-08-28T02:19:47.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47092024-08-28T02:19:47.244ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47102024-08-28T02:19:47.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47112024-08-28T02:19:47.247ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
47122024-08-28T02:19:47.247ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47132024-08-28T02:19:47.247ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fe942cd2-75c5-470d-ad79-083911a726a8", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47142024-08-28T02:19:47.247ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47152024-08-28T02:19:47.250ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47162024-08-28T02:19:47.256ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d zone_address = fd00:1122:3344:101::2
47172024-08-28T02:19:47.256ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47182024-08-28T02:19:47.257ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19251a02-8f65-4124-89b7-c3acee1340de", "content-length": "26", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47192024-08-28T02:19:47.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47202024-08-28T02:19:47.259ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
47212024-08-28T02:19:47.271ZINFOnexus (BackgroundTasks): region snapshot replacement find affected volumes task done background_task = region_snapshot_replacement_step file = nexus/src/app/background/tasks/region_snapshot_replacement_step.rs:439
47222024-08-28T02:19:47.271ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 126.215686ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
47232024-08-28T02:19:47.272ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47242024-08-28T02:19:47.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47252024-08-28T02:19:47.274ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47262024-08-28T02:19:47.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47272024-08-28T02:19:47.310ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47282024-08-28T02:19:47.310ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "22a32c50-ec99-4e42-8cd0-57bc587e4d24", "content-length": "372", "date": "Wed, 28 Aug 2024 02:19:46 GMT"} })
47292024-08-28T02:19:47.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47302024-08-28T02:19:47.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47312024-08-28T02:19:47.313ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47322024-08-28T02:19:47.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47332024-08-28T02:19:47.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47342024-08-28T02:19:47.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47352024-08-28T02:19:47.319ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47362024-08-28T02:19:47.319ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5e8c24ec-4e5c-412e-852c-4eb17a439010", "date": "Wed, 28 Aug 2024 02:19:46 GMT"} })
47372024-08-28T02:19:47.321ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 176.385253ms iteration = 20 status = {}
47382024-08-28T02:19:47.326ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
47392024-08-28T02:19:47.326ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2bef67d5-7228-41c4-9623-e659c17339fa", "content-length": "2", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47402024-08-28T02:19:47.328ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47412024-08-28T02:19:47.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47422024-08-28T02:19:47.330ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 184.896461ms iteration = 18 status = null
47432024-08-28T02:19:47.342ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.019763ms iteration = 18 status = {}
47442024-08-28T02:19:47.400ZDEBGnexus (BackgroundTasks): client request SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47452024-08-28T02:19:47.401ZDEBGnexus (BackgroundTasks): client response SledAgent = 40b49400-5cd0-4d5c-a006-b241a1ca8261 background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "96815cfa-85d6-40a7-b87f-45fa1f178f6a", "content-length": "691", "date": "Wed, 28 Aug 2024 02:19:46 GMT"} })
47462024-08-28T02:19:47.401ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
47472024-08-28T02:19:47.401ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
47482024-08-28T02:19:47.401ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
47492024-08-28T02:19:47.403ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 257.655607ms iteration = 19 status = {}
47502024-08-28T02:19:47.441ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47512024-08-28T02:19:47.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47522024-08-28T02:19:47.444ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: 625c1240-cbb3-4493-85ca-e66fd5998bf2, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60) } rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
47532024-08-28T02:19:47.444ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47542024-08-28T02:19:47.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47552024-08-28T02:19:47.453ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
47562024-08-28T02:19:47.454ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f65e47e1-51a6-4366-8207-49bdbee1b1fc", "content-length": "121", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47572024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
47582024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a864d58d-7bfd-48a1-b0ae-9cb4d4a61b60
47592024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
47602024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
47612024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
47622024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d routes = {}
47632024-08-28T02:19:47.454ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d
47642024-08-28T02:19:47.454ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47652024-08-28T02:19:47.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47662024-08-28T02:19:47.456ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
47672024-08-28T02:19:47.456ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "77eff5e6-60d8-469e-a712-f31ecf3137c2", "content-length": "2", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47682024-08-28T02:19:47.456ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
47692024-08-28T02:19:47.456ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0b5e281f-121f-4dda-8c92-d7ec115b5bcc", "content-length": "51", "date": "Wed, 28 Aug 2024 02:19:47 GMT"} })
47702024-08-28T02:19:47.456ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch_location = Switch0
47712024-08-28T02:19:47.456ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
47722024-08-28T02:19:47.557ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4a8091d7-33cd-4381-8439-e2d7b238fb5e", "date": "Wed, 28 Aug 2024 02:19:46 GMT"} })
47732024-08-28T02:19:47.557ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d switch = Switch0
47742024-08-28T02:19:47.557ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
47752024-08-28T02:19:47.557ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 0c5c13d7-1d6f-4932-bd82-66253028d19d result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6be23edd-e4a1-449d-b335-6c4a6b5ebbad", "content-length": "667", "date": "Wed, 28 Aug 2024 02:19:46 GMT"} })
47762024-08-28T02:19:47.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47772024-08-28T02:19:47.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47782024-08-28T02:19:47.561ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47792024-08-28T02:19:47.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47802024-08-28T02:19:47.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47812024-08-28T02:19:47.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47822024-08-28T02:19:47.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47832024-08-28T02:19:47.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47842024-08-28T02:19:47.574ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 429.468918ms iteration = 18 status = {}
47852024-08-28T02:19:48.124ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edc75e5b-5831-4539-8208-19f270595199 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47862024-08-28T02:19:48.124ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edc75e5b-5831-4539-8208-19f270595199 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47872024-08-28T02:19:48.126ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = edc75e5b-5831-4539-8208-19f270595199 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47882024-08-28T02:19:49.146ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d78e286-24bb-4cdc-ac31-023e154fb72a roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47892024-08-28T02:19:49.147ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d78e286-24bb-4cdc-ac31-023e154fb72a resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47902024-08-28T02:19:49.148ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2473 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8d78e286-24bb-4cdc-ac31-023e154fb72a response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47912024-08-28T02:19:50.168ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 21d0dba6-dcbb-45cc-8795-0e6bdb02adf2 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47922024-08-28T02:19:50.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 21d0dba6-dcbb-45cc-8795-0e6bdb02adf2 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47932024-08-28T02:19:50.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2952 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 21d0dba6-dcbb-45cc-8795-0e6bdb02adf2 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47942024-08-28T02:19:51.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5f141e81-1c9f-4266-90bc-cb9c5ceef431 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47952024-08-28T02:19:51.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5f141e81-1c9f-4266-90bc-cb9c5ceef431 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47962024-08-28T02:19:51.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2752 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 5f141e81-1c9f-4266-90bc-cb9c5ceef431 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47972024-08-28T02:19:52.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9a0d7e01-313f-447c-af12-447a193f6885 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47982024-08-28T02:19:52.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9a0d7e01-313f-447c-af12-447a193f6885 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
47992024-08-28T02:19:52.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2580 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 9a0d7e01-313f-447c-af12-447a193f6885 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48002024-08-28T02:19:53.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 918262e8-8fdd-4588-a793-ad4231469917 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48012024-08-28T02:19:53.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 918262e8-8fdd-4588-a793-ad4231469917 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48022024-08-28T02:19:53.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2359 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 918262e8-8fdd-4588-a793-ad4231469917 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48032024-08-28T02:19:54.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fedfe8ee-8af7-442c-a09b-7faaf1f4460b roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48042024-08-28T02:19:54.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fedfe8ee-8af7-442c-a09b-7faaf1f4460b resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48052024-08-28T02:19:54.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2311 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = fedfe8ee-8af7-442c-a09b-7faaf1f4460b response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48062024-08-28T02:19:55.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 35fe93ee-cd1c-4923-9482-7edd509243dc roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48072024-08-28T02:19:55.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 35fe93ee-cd1c-4923-9482-7edd509243dc resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48082024-08-28T02:19:55.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3411 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 35fe93ee-cd1c-4923-9482-7edd509243dc response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48092024-08-28T02:19:56.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2727 local_addr = [fd00:1122:3344:101::b]:55546 method = GET remote_addr = [fd00:1122:3344:101::d]:58926 req_id = 47cc6d82-61fd-4ca7-8841-58bbffd3cc81 response_code = 200 uri = /50ac3e36-ea5c-4742-b23f-aa0a78804739
48102024-08-28T02:19:56.303ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6d141fe2-681a-4b4c-ba79-27f46076b742 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48112024-08-28T02:19:56.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6d141fe2-681a-4b4c-ba79-27f46076b742 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48122024-08-28T02:19:56.304ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1869 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6d141fe2-681a-4b4c-ba79-27f46076b742 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48132024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
48142024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48152024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
48162024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48172024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48182024-08-28T02:19:57.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48192024-08-28T02:19:57.147ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48202024-08-28T02:19:57.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48212024-08-28T02:19:57.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48222024-08-28T02:19:57.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48232024-08-28T02:19:57.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48242024-08-28T02:19:57.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48252024-08-28T02:19:57.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48262024-08-28T02:19:57.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48272024-08-28T02:19:57.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48282024-08-28T02:19:57.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4dbaa16e-087a-4bfd-b449-5e4a2238b758, lookup_type: ById(4dbaa16e-087a-4bfd-b449-5e4a2238b758) } result = Ok(())
48292024-08-28T02:19:57.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48302024-08-28T02:19:57.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48312024-08-28T02:19:57.165ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 original_time_created = 2024-08-28 02:10:55.235565 UTC target_id = 4dbaa16e-087a-4bfd-b449-5e4a2238b758 time_created = 2024-08-28 02:10:55.235565 UTC
48322024-08-28T02:19:57.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.774603ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4dbaa16e-087a-4bfd-b449-5e4a2238b758","time_created":"2024-08-28T02:10:55.235565Z"}
48332024-08-28T02:19:57.174ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.916194ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48342024-08-28T02:19:57.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3158794d-5476-4114-9125-40fcfe942f41 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48352024-08-28T02:19:57.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3158794d-5476-4114-9125-40fcfe942f41 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48362024-08-28T02:19:57.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1901 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 3158794d-5476-4114-9125-40fcfe942f41 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48372024-08-28T02:19:58.343ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 58b9a606-88c4-406c-932c-c425839e2c4d roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48382024-08-28T02:19:58.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 58b9a606-88c4-406c-932c-c425839e2c4d resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48392024-08-28T02:19:58.346ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2570 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 58b9a606-88c4-406c-932c-c425839e2c4d response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48402024-08-28T02:19:59.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6717cbaa-f4c3-4cf8-8df6-31cbec43dbdb roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48412024-08-28T02:19:59.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6717cbaa-f4c3-4cf8-8df6-31cbec43dbdb resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48422024-08-28T02:19:59.366ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2693 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 6717cbaa-f4c3-4cf8-8df6-31cbec43dbdb response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48432024-08-28T02:20:00.385ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8dc05e9c-2d90-4228-889f-e198ceeffdf7 roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48442024-08-28T02:20:00.385ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8dc05e9c-2d90-4228-889f-e198ceeffdf7 resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48452024-08-28T02:20:00.387ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1769 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 8dc05e9c-2d90-4228-889f-e198ceeffdf7 response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48462024-08-28T02:20:01.406ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69980545-d45b-4c55-b9fc-a6781ebc47ec roles = RoleSet { roles: {} } uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48472024-08-28T02:20:01.407ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69980545-d45b-4c55-b9fc-a6781ebc47ec resource = Database result = Ok(()) uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261
48482024-08-28T02:20:01.408ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2213 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36761 req_id = 69980545-d45b-4c55-b9fc-a6781ebc47ec response_code = 200 uri = /probes/40b49400-5cd0-4d5c-a006-b241a1ca8261